Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handling fixed column relationships by specific_combinations and SpecificCombinationTransformer. #236

Merged
merged 8 commits into from
Nov 18, 2024

Conversation

jalr4ever
Copy link
Collaborator

@jalr4ever jalr4ever commented Nov 15, 2024

Description

I haved added a new SpecificCombinationTransformer, and modifed FixedCombinationTransformer. And a test case for SpecificCombinationTransformer.

Motivation and Context

SpecificCombinationTransformer would transforming when user specified some column-mapping combination, and FixedCombinationTransformer would not running at the same time.

I make FixedCombinationTransformer as default column-mapping combination detector and transformer.As for SpecificCombinationTransformer, it would transforming when user specified some column-mapping combinations, make sure the sample data column-mapping adhere to the specified combinations. At the same time, FixedCombinationTransformer would not run transforming when SpecificCombinationTransformer activated.

How has this been tested?

See test case.

Types of changes

  • Maintenance (no change in code, maintain the project's CI, docs, etc.)
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.

@jalr4ever jalr4ever marked this pull request as ready for review November 15, 2024 10:22
@Wh1isper
Copy link
Collaborator

Do we really need to distinguish between is_been_specified and is_exist_fixed_combinations, seems just for log... Or do you think it would be beneficial downstream?

@jalr4ever
Copy link
Collaborator Author

Do we really need to distinguish between is_been_specified and is_exist_fixed_combinations, seems just for log... Or do you think it would be beneficial downstream?

@Wh1isper Yes we need.

Currently, the consideration is that when the shape of the data does not meet expectations, it can be determined from the logs what caused the lack of transformation; on the other hand, there may be other combinations in the future, and different scenarios may have different logic.

@codecov-commenter
Copy link

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

Attention: Patch coverage is 88.42105% with 11 lines in your changes missing coverage. Please review.

Project coverage is 82.75%. Comparing base (00685a3) to head (3c16c52).
Report is 14 commits behind head on main.

Files with missing lines Patch % Lines
.../data_processors/transformers/fixed_combination.py 76.92% 9 Missing ⚠️
...ta_processors/transformers/specific_combination.py 96.42% 2 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #236      +/-   ##
==========================================
+ Coverage   82.17%   82.75%   +0.57%     
==========================================
  Files          84       88       +4     
  Lines        4146     4470     +324     
==========================================
+ Hits         3407     3699     +292     
- Misses        739      771      +32     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Wh1isper
Copy link
Collaborator

Wh1isper commented Nov 15, 2024

@jalr4ever Ok and I suggest replace is_exist_fixed_combinations with @property decorator

Like this:

@property
def is_exist_fixed_combinations(self) -> bool:
  return bool(self.is_exist_fixed_combinations)

Then we can reduce the code in fit function, no need tow if blocks but one is enough

@jalr4ever jalr4ever requested a review from Wh1isper November 15, 2024 11:23
Co-authored-by: Zhongsheng Ji <9573586@qq.com>
@jalr4ever jalr4ever requested a review from Wh1isper November 15, 2024 11:30
Copy link
Collaborator

@Wh1isper Wh1isper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jalr4ever jalr4ever merged commit 314ada9 into main Nov 18, 2024
12 checks passed
@jalr4ever jalr4ever deleted the jalr4ever-specific-combination-transform branch November 21, 2024 07:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants