Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(sampler): add min-max sampler function #19279

Merged
merged 2 commits into from
Nov 27, 2023

Conversation

snukhulov
Copy link
Contributor

@snukhulov snukhulov commented Nov 6, 2023

Brief Information

This pull request is in the type of:

  • bug fixing
  • new feature
  • others

What does this PR do?

Add min-max sampler function

Details

Before: What was the problem?

In some cases min-max sampling algorithm fits line data more naturally than the existing ones. For example, for a sine with a lot of data points min-max sampler is the best choice.

Example.
The original sine without sampling (200 000 data points).
Снимок экрана 2023-11-07 в 00 42 48

Using 'average' sampler
Снимок экрана 2023-11-07 в 00 44 15

Using 'lttb' sampler
Снимок экрана 2023-11-07 в 00 43 58

Using 'max' sampler
Снимок экрана 2023-11-07 в 00 43 39

Using 'min' sampler
Снимок экрана 2023-11-07 в 00 43 18

After: How does it behave after the fixing?

Using new 'minmax' sampler
Снимок экрана 2023-11-07 в 01 08 29

Document Info

One of the following should be checked.

Misc

ZRender Changes

  • This PR depends on ZRender changes (ecomfe/zrender#xxx).

Related test cases or examples to use the new APIs

N.A.

Others

Merging options

  • Please squash the commits into a single one when merging.

Other information

Copy link

echarts-bot bot commented Nov 6, 2023

Thanks for your contribution!
The community will review it ASAP. In the meanwhile, please checkout the coding standard and Wiki about How to make a pull request.

Document changes are required in this PR. Please also make a PR to apache/echarts-doc for document changes and update the issue id in the PR description. When the doc PR is merged, the maintainers will remove the PR: awaiting doc label.

Copy link
Contributor

github-actions bot commented Nov 7, 2023

The changes brought by this PR can be previewed at: https://echarts.apache.org/examples/editor?version=PR-19279@4a63340

@echarts-bot echarts-bot bot added PR: doc ready and removed PR: awaiting doc Document changes is required for this PR. labels Nov 7, 2023
Copy link
Contributor

@Ovilia Ovilia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to be a great idea. Can you explain what min-max mean first?

@snukhulov
Copy link
Contributor Author

snukhulov commented Nov 8, 2023

This seems to be a great idea. Can you explain what min-max mean first?

Thank you.
Like 'min' sampler that highlights an extremum (turning-point) at a local minimum and like 'max' sampler that highlights an extremum at a local maximum, 'minmax' sampler highlights the larger extremum (minimum or maximum extremum in absolute comparison) at a frame of data points.
'minmax' sampler is suitable for sine or other periodic data

@Ovilia
Copy link
Contributor

Ovilia commented Nov 15, 2023

Thanks for your explanation. Can you provide a link or something for further understanding? Is this a concept well-known or you invented it?

@snukhulov
Copy link
Contributor Author

snukhulov commented Nov 19, 2023

Thanks for your explanation. Can you provide a link or something for further understanding? Is this a concept well-known or you invented it?

It's my own implementation of the known min-max downsampling algorithm

Copy link
Contributor

@Ovilia Ovilia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Please also add test cases in test/line-sampling.html. Please pay special attention both to the cases where there are negative values or not.

@snukhulov
Copy link
Contributor Author

Thanks! Please also add test cases in test/line-sampling.html. Please pay special attention both to the cases where there are negative values or not.

I added the described test cases

Copy link
Contributor

@Ovilia Ovilia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution! This will be released in the next version.

@Ovilia
Copy link
Contributor

Ovilia commented Nov 27, 2023

Updated demo here

@Ovilia Ovilia added this to the 5.5.0 milestone Nov 27, 2023
@Ovilia Ovilia merged commit c5e0910 into apache:master Nov 27, 2023
2 checks passed
Copy link

echarts-bot bot commented Nov 27, 2023

Congratulations! Your PR has been merged. Thanks for your contribution! 👍

@sulivanganter
Copy link

Hi, @snukhulov

I think there might be something wrong. If you change the number of samples in your example to 100,000, updated exemple ,you can see the problem with returning just the absolute max value.

This is just a suggestion and has not been tested:

You need two values for each frame for min and max, something like this:

 minmax: function (frame) {
    let turningPointMaxValue = -Infinity;
    let turningPointMinValue = Infinity;

    for (let i = 0; i < frame.length; i++) {
        const originalValue = frame[i];

        if (originalValue > turningPointMaxValue) {
            turningPointMaxValue = originalValue;
        }
        
        if (originalValue < turningPointMinValue) {
            turningPointMinValue = originalValue;
        }
    }

    // Return an array with the min and max values
    return [turningPointMinValue, turningPointMaxValue];
}

@snukhulov
Copy link
Contributor Author

snukhulov commented May 25, 2024

Hi, @snukhulov

I think there might be something wrong. If you change the number of samples in your example to 100,000, updated exemple ,you can see the problem with returning just the absolute max value.

This is just a suggestion and has not been tested:

You need two values for each frame for min and max, something like this:

 minmax: function (frame) {
    let turningPointMaxValue = -Infinity;
    let turningPointMinValue = Infinity;

    for (let i = 0; i < frame.length; i++) {
        const originalValue = frame[i];

        if (originalValue > turningPointMaxValue) {
            turningPointMaxValue = originalValue;
        }
        
        if (originalValue < turningPointMinValue) {
            turningPointMinValue = originalValue;
        }
    }

    // Return an array with the min and max values
    return [turningPointMinValue, turningPointMaxValue];
}

Hi! Thank you for the research and the suggestion!

You're right about there are some problems with the proposed min-max sampler function. It's designed for a sine periodic data like y(x)=A*sin(x+n), but there are some sampling problems with aperiodic component of a signal (like y(x)=A*sin(x+n)+k).

Your suggestion looks great, but I have some doubts about returning type of your function. I think it only could be number, but couldn't be array of numbers.

I thought about improvement of the proposed min-max sampler, but there is motivation to keep it simple like other existing sampling functions.

But I'am still looking for an improvement of min-max sampler function that could solve sampling problem of data with aperiodic component like y(x)=A*sin(x+n)+k and could be simple at the same time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants