Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement version of unaryminus that allows overflow for Spark (#1339) #19

Closed
wants to merge 3 commits into from
Closed

Implement version of unaryminus that allows overflow for Spark (#1339) #19

wants to merge 3 commits into from

Commits on Aug 10, 2021

  1. Implement Spark's variadic HASH() function 

    Summary:
Spark uses a variant of Murmur3; I've copied the hash implementation directly
from the Spark codebase, modified of course for C++.

The Spark algorithm for combining hashes is simple: initialize the hash to 42,
and then use the current hash as the seed for the next hash. NULLs are skipped.
This implementation processes one column at a time.

Differential Revision: D29743513

fbshipit-source-id: 2784e0959183559bf1bafec59e588a290986aa65
    @funrollloops @facebook-github-bot
    funrollloops authored and facebook-github-bot committed Aug 10, 2021
    Configuration menu
    Copy the full SHA
    77a8b4c View commit details
    Browse the repository at this point in the history

  2. Don't depend on "common" registrations in sparksql directory 

    Summary:
Function registration is moved from FunctionBaseTest's constructor to
SetUpTestCase(), which is once-per-process.

Differential Revision: D30061756

fbshipit-source-id: 0ce37ef8a4b0686f70a6fd7395a8b979721672e3
    @funrollloops @facebook-github-bot
    funrollloops authored and facebook-github-bot committed Aug 10, 2021
    Configuration menu
    Copy the full SHA
    a6fb27d View commit details
    Browse the repository at this point in the history

  3. Implement version of unaryminus that allows overflow for Spark (#19) 

    Summary:
Pull Request resolved: #19

Pull Request resolved: facebookexternal/f4d#1339

Reviewed By: pedroerp

Differential Revision: D30092234

fbshipit-source-id: 3a03cd0660bb39be4f60709818048fc9227cf679
    @funrollloops @facebook-github-bot
    funrollloops authored and facebook-github-bot committed Aug 10, 2021
    Configuration menu
    Copy the full SHA
    155c32c View commit details
    Browse the repository at this point in the history