Skip to content
This repository has been archived by the owner on Sep 18, 2023. It is now read-only.

[NSE-245]Adding columnar RDD cache support #246

Merged
merged 4 commits into from
Apr 29, 2021
Merged

[NSE-245]Adding columnar RDD cache support #246

merged 4 commits into from
Apr 29, 2021

Commits on Apr 15, 2021

  1. Adding columnar RDD cache support 

    Signed-off-by: Chendi Xue <chendi.xue@intel.com>
    @xuechendi
    xuechendi committed Apr 15, 2021
    Configuration menu
    Copy the full SHA
    8c75250 View commit details
    Browse the repository at this point in the history

Commits on Apr 21, 2021

  1. Directly save reference, only convert to Array[Byte] when calling by … 

    …BlockManager

Signed-off-by: Chendi Xue <chendi.xue@intel.com>
    @xuechendi
    xuechendi committed Apr 21, 2021
    Configuration menu
    Copy the full SHA
    593d095 View commit details
    Browse the repository at this point in the history

Commits on Apr 22, 2021

  1. Add DeAllocator to construction to make sure this instance will be re… 

    …leased once it be deleted by JVM

Signed-off-by: Chendi Xue <chendi.xue@intel.com>
    @xuechendi
    xuechendi committed Apr 22, 2021
    Configuration menu
    Copy the full SHA
    1ccbc17 View commit details
    Browse the repository at this point in the history

Commits on Apr 28, 2021

  1. Delete cache by adding a release in InMemoryRelation 

    Since unpersist only delete RDD object, seems our deAllocator wasn't being called along
Now we added a release function in InMemoryRelation clearCache() func, may need to think
a new way for 3.1.0

Signed-off-by: Chendi Xue <chendi.xue@intel.com>
    @xuechendi
    xuechendi committed Apr 28, 2021
    Configuration menu
    Copy the full SHA
    36622c2 View commit details
    Browse the repository at this point in the history