Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Java] fast object copy framework in fury java #1679

Open
chaokunyang opened this issue Jun 11, 2024 · 0 comments
Open

[Java] fast object copy framework in fury java #1679

chaokunyang opened this issue Jun 11, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@chaokunyang
Copy link
Collaborator

chaokunyang commented Jun 11, 2024 •
edited
Loading

Is your feature request related to a problem? Please describe.

Deep copy is common in programing to avoid modify existing data, which may introduce side effect and subtle bugs.

Currently fury doesn't support copy natively, users need to serialized data into binary, then deserialize it into object. This is costly, it introduce extra copy, and for immutable objects such as string, time objects, boxed objects, there is no need to copy.

Describe the solution you'd like

We should support copy natively in fury:

  • Design a copy interface
  • MakeSerializer implement the copy interface, throw UnsupportedException by default.
  • For non-jit serializer, override the copy interface to implement copy
  • For immutable object such as String, java.time, just return itself
  • For mutable object, create new object and set all attributes
  • For pojo/bean/record object, implement the copy in a separate class, and forward the copy to that class to reuse the copy implementation in ObjectSerializer/CompatibleObjectSerializer

Additional context

#1014
@chaokunyang chaokunyang added the enhancement New feature or request label Jun 11, 2024
@chaokunyang chaokunyang mentioned this issue Jun 11, 2024
Open
@chaokunyang chaokunyang changed the title [Java] fast object copy natively in fury java [Java] fast object copy framework in fury java Jun 11, 2024
@zhaommmmomo zhaommmmomo mentioned this issue Jun 26, 2024
Merged
2 tasks
chaokunyang pushed a commit that referenced this issue Jul 18, 2024
@zhaommmmomo
feat(java): fast object copy framework in fury java (#1701)
64c995d 
## What does this PR do?

Object deep copy framework in fury java. This PR is only for non-jit
serializer

- For immutable object such as `String`、`java.time`、`int`、`byte`...
return itself.
- For mutable object, create new object and set all attributes.
- For `pojo` / `bean` object, Use `ObjectSerializer.copy(obj)` to create
a new object
- For `Arrays`、`Collection`、`Map` object, create new object, traverse
the elements, call `fory.copy(obj)` to generate new elements and set
them to the new object. Some serializers have special `copy` methods.



## Related issues

[[Java] fast object copy framework in fury java
#1679](#1679)



## Does this PR introduce any user-facing change?

- [x] Does this PR introduce any public API change?
- [ ] Does this PR introduce any binary protocol compatibility change?



1. Provide `fury.copy(obj)` interface, which can deep copy the object
without serialize / deserialize. For example:

```java
User user = new User();
user.setName("a");

// If object contain circular references, please enable copy ref tracking by FuryBuilder#withCopyRefTracking(true) 
Fury fury = Fury.builder().withLanguage(Language.JAVA).withCopyRefTracking(true).build();
User newUser = fury.copy(user);
```



2. Provide `FuryCopyable<T>` interface, which can customize the copy
method of object. For example:

```java
public class User implements Serializable, FuryCopyable<User> {
  
  @OverRide
  public User copy(Fury fury) {
    // do something
    System.out.println("object custom copy method");
    return newUser;
  }
}
```





## Benchmark
> Device

windows 10、12 cores、24G

> JDK

java version "1.8.0_202"
Java(TM) SE Runtime Environment (build 1.8.0_202-b08)
Java HotSpot(TM) 64-Bit Server VM (build 25.202-b08, mixed mode)

java version "17.0.8" 2023-07-18 LTS
Java(TM) SE Runtime Environment (build 17.0.8+9-LTS-211)
Java HotSpot(TM) 64-Bit Server VM (build 17.0.8+9-LTS-211, mixed mode,
sharing)

> Test data

see [benchmark
code](https://github.com/zhaommmmomo/fury/tree/copy_benchmark/java/fury-benchmark/src/main/java/org/apache/fury/benchmark)
```java
int size = 128;
int[] intArr = new int[size];
List<Object> list = new ArrayList<>(size);
Map<Object, Object> map = new ConcurrentHashMap<>();
BeanA beanA = BeanA.createBeanA(size); // org.apache.fury.test.bean.BeanA

for (int i = 0; i < size; i++) {
  intArr[i] = i;
  list.add(i);
  map.put(i, UUID.randomUUID().toString());
}
```

> Test

**JMH**:
benchmarkMode({Mode.Throughput})、fork(1)、threads(1)、warmup(iterations =
3, time = 1)、measurement(iterations = 5, time =
5)、outputTimeUnit(TimeUnit.SECONDS)
*JDK8*


*JDK17*


**JMH**:
benchmarkMode({Mode.AverageTime})、fork(1)、threads(1)、warmup(iterations =
3, time = 1)、measurement(iterations = 5, time =
5)、outputTimeUnit(TimeUnit.MILLISECONDS)
**Each benchmark method will be looped 10,000 times**
*JDK8*


*JDK17*
This was referenced Jul 20, 2024
Open
Closed
Closed
Closed
Closed
Merged
Closed
chaokunyang added a commit that referenced this issue Jul 24, 2024
@chaokunyang
chore(java): rename copyTrackingRef to copyRef (#1748)
7b6e9ed 
## What does this PR do?

rename copyTrackingRef to copyRef

## Related issues
#1679 
#1747


## Does this PR introduce any user-facing change?

<!--
If any user-facing interface changes, please [open an
issue](https://github.com/apache/fury/issues/new/choose) describing the
need to do so and update the document if necessary.
-->

- [ ] Does this PR introduce any public API change?
- [ ] Does this PR introduce any binary protocol compatibility change?


## Benchmark

<!--
When the PR has an impact on performance (if you don't know whether the
PR will have an impact on performance, you can submit the PR first, and
if it will have impact on performance, the code reviewer will explain
it), be sure to attach a benchmark data here.
-->
@chaokunyang chaokunyang mentioned this issue Jul 27, 2024
Merged
2 tasks
chaokunyang added a commit that referenced this issue Jul 27, 2024
@chaokunyang
feat(java): support deep ref copy (#1771)
fd39760 
## What does this PR do?

support deep ref copy

## Related issues

Closes #1747
#1679


## Does this PR introduce any user-facing change?

<!--
If any user-facing interface changes, please [open an
issue](https://github.com/apache/fury/issues/new/choose) describing the
need to do so and update the document if necessary.
-->

- [ ] Does this PR introduce any public API change?
- [ ] Does this PR introduce any binary protocol compatibility change?


## Benchmark

<!--
When the PR has an impact on performance (if you don't know whether the
PR will have an impact on performance, you can submit the PR first, and
if it will have impact on performance, the code reviewer will explain
it), be sure to attach a benchmark data here.
-->
@zhaommmmomo zhaommmmomo mentioned this issue Aug 7, 2024
Merged
2 tasks
chaokunyang added a commit that referenced this issue Aug 13, 2024
@zhaommmmomo @chaokunyang
feat(java): Support copy capabilities for some classes without no-arg…
9a39cb3 
…ument constructors (#1794)

## What does this PR do?
Some classes with no-argument constructors will report an error when
calling `copy()`.

This pr:
- implement the copy method for the no-argument constructor serializer
- add test case


## Related issues
#1777
#1679

## Does this PR introduce any user-facing change?

<!--
If any user-facing interface changes, please [open an
issue](https://github.com/apache/fury/issues/new/choose) describing the
need to do so and update the document if necessary.
-->

- [ ] Does this PR introduce any public API change?
- [ ] Does this PR introduce any binary protocol compatibility change?


## Benchmark

<!--
When the PR has an impact on performance (if you don't know whether the
PR will have an impact on performance, you can submit the PR first, and
if it will have impact on performance, the code reviewer will explain
it), be sure to attach a benchmark data here.
-->

---------

Co-authored-by: Shawn Yang <chaokunyang@apache.org>
@zhaommmmomo zhaommmmomo mentioned this issue Aug 15, 2024
Merged
2 tasks
chaokunyang pushed a commit that referenced this issue Aug 15, 2024
@zhaommmmomo
feat(java): support jdk9+ java.util.ImmutableCollections copy (#1800)
bfa6dea 
## What does this PR do?
support jdk9+ java.util.ImmutableCollections copy

## Related issues
#1679


## Does this PR introduce any user-facing change?
- [ ] Does this PR introduce any public API change?
- [ ] Does this PR introduce any binary protocol compatibility change?


## Benchmark
@zhaommmmomo zhaommmmomo mentioned this issue Aug 22, 2024
Open
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@chaokunyang