-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
High RAM consumption during filter updates #5606
Comments
Confirm. Have the same problem. With all functions disabled as DHCP, optimistic caching and all disabled filters i got killed process and "out of memory" too. |
Hello.
Just to clarify, are you using the file paths instead of URIs in your filtering settings? Also, how large are the rule files exactly? |
TL;DR
I am using the URI, all from AdGuard: What I meant to say is the working folder is /opt and not /var or /tmp, trying to save some RAM.
Getting details for this question, I found more data that I hope is useful. For some reason, AGH allocates a lot of memory when changing settings, like "Filter update interval" (even if disabling). Also, simply disabling lists can make it crash too, because it seems to also allocate a lot of memory when a list is disabled...
Free RAM values are a loose reference, as I noticed they float a lot as I change settings in AGH for testing.
With only this list (9.5 MB) any changes to settings, like "filter update interval", makes AGH crash...
No crash when changing settings...
No crash when changing settings, but free RAM drops to around 40 MB for a moment. I think something is wrong with how much RAM is allocated just to change one setting (like filter update interval)... |
Thanks for the details. There may be a few issues with how settings are applied, but it would be hard to diagnose without collecting memory profiles by setting Some other things that you can try:
|
OK, when I have some time I will collect the debug data and also test the GOMEMLIMIT variable, then report here. EDIT: EDIT: |
log1 had GOGC=40 and GOMEMLIMIT=100MiB and changed update interval to see free RAM dropping; |
Thanks for testing our the environment variables and attaching the profiles! Unfortunately, the profiles show that most memory is consumed by the filtering engine, so there probably won't be a way to decrease that without a proper rewrite/optimization of the In the meantime, if the env solution works for you, it's probably the best you can do, along with the filepath-based filters. |
I guess, without looking into code, it makes sense that updating filters list temporarily use a lot of memory building a new rules list before replacing the old one, but I don't see why just changing the update interval would require so much memory to be allocated... That's what I tried to log. On the filepath-based filters, it seems that it still makes copies of local filters lists. I even made it with the exact name and local it is saved, and refreshing makes it overwrite the list with itself. Prabably that could be improved, not making copies of local lists, because it causes unnecessary writing to the router flash...
|
I've created #5629 to see if we can remove the copying. Thanks for pointing that out. |
|
You would have to run your own external script for updating those local files. Also, similar to
so if you have an init script for starting adguardhome, you would place the env arguments as pre-commands or pre-launch arguments. |
Thanks for the confirmation on the local files. Wrt the OpenWrt service, adding
|
Yep for openwrt, that works perfectly. For Entware or other router implementations it might have to be adapted per setup. For openwrt, procd takes care of the "logic" as long as you follow openwrts template. For Entware, it would have to be listed as a "pre-arg". |
I have the following command running once a week in the router: where /opt/AdGuardHome/agh_filters_list is a text file with the filters url one per line, like this: edit: I forgot to mention I stop and restart AdGuard before and after updating the lists, because of the memory crash and also to make the list (re)load. |
I went much simpler and have a list of Doing this once a day has halved the memory agh uses, with no router crashes so far. |
I added a GoodbyeAds, on Docker with 1 core 512mb memory, updating rules or custom rules will inevitably lead to freezes or crashes, the log does not contain any errors. |
Same issue. When the filters too much, the AdGuardHome processes crashes by OOM.
|
How to correctly write this in a systemd service file? Is this correct?
//update Yep. I lost access to the device after a week. Once a week, adguard updates its lists. How to properly limit the memory for a go application using systemd? //update2 Any help? |
The unit has a EnvironmentFile definition so you can simply put the variables into GOGC=40 and restart the service # systemctl restart AdGuardHome.service You can validate if it worked with something like this # cat /proc/$(pgrep AdGuardHome)/environ | tr '\0' '\n' | grep ^GO
GOGC=40 The |
To summarize how to: Edit file by:
add line after:
Save and reboot |
Hello, did you manage to solve the problem? My router ax3600 crashes as soon as I run adguard |
Yes this solution works for me and my friend with AX3600 router.
Follow the steps which I made and reboot. After refresh of Adguard it
should works.
just to make sure have you followed this installation on Openwrt?
https://openwrt.org/docs/guide-user/services/dns/adguard-home
Without that it will not work properly.
I keep fingers crossed
Karol
…On Sat, 16 Mar 2024 at 13:43, marcel13000 ***@***.***> wrote:
Hello, did you manage to solve the problem? My router crashes as soon as I
run adguard
—
Reply to this email directly, view it on GitHub
<#5606 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AXRAE5NUDMUUDKQX5SJM5YTYYQ45VAVCNFSM6AAAAAAV7EZ7SKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBRHE3TMMBRGM>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Keep your ad block list small. Maybe start with two and test. Edit: Here's the line I have in /etc/init.d/adguardhome: |
@ainar-g |
There is no update. AdGuard Home needs RAM to keep and update filtering-rule engines. The larger your lists are, the more RAM is necessary. I'm not sure if there's anything to do in this issue,so I'm going to close it. |
@privacy-advo please check my comment [on Mar 6] and add one line: Edit file by:
add line in above file after linke:
Save and reboot Then your router will not fall |
For me setting |
Prerequisites
I have checked the Wiki and Discussions and found no answer
I have searched other issues and found no duplicates
I want to report a bug and not ask a question
Operating system type
Linux, OpenWrt
CPU architecture
AMD64
Installation
GitHub releases or script from README
Setup
On a router, DHCP is handled by the router
AdGuard Home version
v0.108.0-a.466+9f7a582d
Description
What did you do?
Installed AGH on router and set filters to update every 24h.
Filters are saved in /opt (not RAM) trying to use less RAM, also logs are reduced to spare RAM.
Expected result
While using around 250MB of RAM for AdGuardHome only, the router usually still has more than 70 MB of free RAM.
AdGuardHome should manage the available RAM to update filters without crashing.
Actual result
"top" command shows that AGH is reserving 932 MB when there are 406 MB usable (229% of VSZ).
When comes the time to update filters, AGH tries to allocate 932 MB and then it invokes oom-killer on itself due to lack of free RAM.
Screenshots (if applicable)
Additional information
Right now it's running edge build, but it also happens with beta build and with v107.
The problem seems to be AGH trying to blindly allocate 930 MB of RAM when it should instead check how much free RAM there is (more than 70 MB) and manage to update the filters with it.
I suppose 70 MB should be plenty RAM to update any lists.
The text was updated successfully, but these errors were encountered: