Processor load

[dl2ki]

QLog generates a high processor load when DX-Cluster is switched on. With 'telnet.reversebeacon.net:7300' I measured up to 25%.
With the same settings it is with CQRLog up to approx. 6%.

[foldynl]

does it appear after a long time or as soon as QLog is connected to DXC?
do you see peaks or the load is the same during a time period?

[foldynl]

Have you set "Spot Aging" time in Bandmap or you have set "Clear older than" to "never" in Bandmap widget?

[dl2ki]

Unfortunately, I did not note down the spot rate during the measurement. You could make up for it at the next contest, hi.

does it appear after a long time or as soon as QLog is connected to DXC?
do you see peaks or the load is the same during a time period?

The high processor load only occurs after some time when the spot rate is high. The value then remains relatively constant in the range of 25%. Right now the spot rate is around 16.322. The processor load is around 1% to 4%.

I used the telnet DX cluster "telnet.reversebeacon.net:7300", because it gives the best results for CW operation. The total number of automatic sports is of course high here.

In CQRLog I also use the function "Callsign alert", because I can see relatively fast, where known radio partners are active.

Have you set "Spot Aging" time in Bandmap or you have set "Clear older than" to "never" in Bandmap widget?

The spot aging period is set to "Clear older than 30 Minutes". This could certainly be set shorter for high spot rates.

In CQRLog at high spot rates the lines in the Telnet DX cluster window are not displayed completely after some time and then the DX cluster window freezes completely. This is not the case in QLog.

Saku has already improved the behavior a bit, but could not solve the problem completely.

Apparently the processing of the DX-Cluster spots generally requires a high processor load.

[foldynl]

I have identified following bottlenecks in DXC processing

1. QLog queries a database for every received spot - to determine Callsign status (new, worked etc).
2. it takes some time when QLog refreshes the bandmap widget - the bandmap is refreshed for every spot on the displayed band.
3. Band Aging takes some time when Band aging period is "long" - it means there are many QSOs on the bandmap.

At this moment, I have no quick solution for it - the fix will not be a part of v0.9.

[foldynl]

In CQRLog I also use the function "Callsign alert", because I can see relatively fast, where known radio partners are active.

I have forgotten, Callsign alerting will be implemented in the next release (hopefully)

[foldynl]

I can see that you have tested v0.9.0.

do you see any improvement in CPU load?

[dl2ki]

Hi,

I have only had a quick look at testing_0.9 so far. The processor load is currently at 1% to 4%. But the spot rate is currently only about 10,200 spots per hour.

You would have to check this once at the weekend when the spot rates are much higher.

[dl2ki]

At the moment, RBN has a spot rate of about 61,600 spots/h. The CPU load fluctuates between approx. 5% and 12%. You can't see why the load fluctuates so differently. A better assessment is not possible at the moment. It would be necessary to wait for higher spot rates.

I use a filter here: CW, all HF bands, all continents, spotter continent Europe only. This reduces the lines displayed in the DX Cluster widget.

If I now reduce the spots to one band, e.g. 80m, the lines in the DX Cluster widget will be reduced even more.

However, this does not reduce the CPU load.

So my question: Are the database queries done for each spot received by RBN before the filter, or only for the filtered spots? If you put the database queries behind the filter, should it be possible to reduce the number of database queries? At least for those who use filters.

Or do I have a thinking error here?

[foldynl]

Many thanks for your report. I have also tested the CPU Load. I used FT8 RBN (it produces also a huge traffic) and it looked good. My problem was, apparently my CPU is powerful than yours. To simulate your load, I had to change the CPU profile to minimum. And then I didn't reach the reported load. That's why I was curious about your measurements.

To your question. The filter has no impact on received spots. It means that all spots are processes and stored in memory by QLog because when you want to change the filter, QLog has to show an old spots. Therefore spots cannot be filtered out immediately after receive function.

To improve a performance, I added Memory Caches and other improvements. What I found out that the database is not such a bottleneck. There was an issue with redrawing the bandmap. The original QLog redrew the whole bandmap after EVERY received spot. Currently, Qlog redraw only the necessary elements. Unfortunately, if there is a large number of spots on the map, then redrawing again takes some CPU time.

But I admit, it is necessary to look at it again later, because especially a memory management is not completely under control now. 61000 spots/h is OK when qlog runs for a few hours. However, if it runs, for example, a day, I think it could be a problem.

[dl2ki]

yes, it is certainly the case that my CPU is not the fastest here. But I think that you can still work very well with QLog. The CPU load in this range is not a problem.

Here is a short overview of my system for your information.

[dl2ki]

Hi,

This morning the spot rate was about 145000 spots/minute. Therefore I checked the CPU load again.

In the DX-Cluster these filters are used:

After starting QLog and DX-Cluster Connect the CPU load is about 9% to 17%. After one minute at about 21% and after 2 minutes at about 25%.

After about 4 minutes the bandmap buttons 'Clear All, In, Out' react sluggish. The same applies to the context menus in the DX-Cluster widget and in the bandmap.

After that, the bandmap buttons respond more and more sluggishly, and then freeze longer. The QLog menu 'File, Logbook, Sevice ...' also freezes.

The DX cluster always runs at the same speed.

Once QLog reacted after some time to the Disconnect button in the DX-Cluster widget and the behavior returned to normal.

The freezing affects the whole KDE user interface, so I restarted the PC with reboot.

I can't tell for sure right now if QLog is freezing, or if it's just the response time to controls that is extremely slowed down.

The CPU load does not rise above 25% here.

With CQRLog, the CPU load remains in the 6% to 8% range.

Unfortunately, QLog is currently not usable at high spot rate.

[foldynl]

Unfortunately, I can confirm your observation with one difference, I'm able to make it working, but my CPU (i7-11800H @ 2.30GHz) is 80-90% busy.

Despite the fact, that you have set DXC Filter, QLog receives and handle all spots. It is important to know that only filtered QSOs are shown but all spots are processed by QLog (Qlog has to determine DXCC country, Logbook status -worked/new, determine Spotter DXCC and country and many other parameters for EVERY received spot, Based on these computed parameters, QLog can provide an extended DXC filter).

QLog has never been designed to handle RBN traffic during CQ WW WPX Contest CW where a spot rate is over 100k spots/h (btw it means around 30 spots/s in average - with high peaks)

if we would prepare QLog for this traffic, it would mean redesign the logic of QLog DXC processing. It would probably be necessary to use separate processes/threads for parsing QSOs, processing QSOs, repainting of bandmap, sending a notificaiton via Network etc. Currently, all these functions are done in one thread.

I'll leave it as-in now. But if there will be a Pull Request that fixes/improves it, I would like to welcome it.

[dl2ki]

Hi,

I'll leave it as-in now. But if there will be a Pull Request that fixes/improves it, I would like to welcome it.

Yes, a good idea ...

The high spot rate does not affect the system in general. You can replace RBN with a normal DX cluster in these cases. In these contest conditions RBN makes no sense anyway.

From my point of view this is no problem for the use of QLog.

[foldynl]

I am closing the issue. Currently not observed the high CPU usage