Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add truncation filter strategy for all filters #180

Merged
merged 2 commits into from
Dec 19, 2024

Conversation

JessieAMorris
Copy link
Contributor

Description

I have a use case where I want to filter out specific strings only leveraging the identifier filter. However, I'd like a LAST_4 style filter strategy for the identifier filter. The truncate filter more or less did what I wanted, but it was previously only supported for the zip filter. It also only supported leaving leading characters.

This change adds support for TRUNCATE on the other filters and adds a truncateDirection for specifying if leading or trailing characters should be left.

Issues Resolved

List any issues this PR will resolve, e.g. Closes [...].

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@@ -107,6 +110,18 @@ public abstract class AbstractFilterStrategy {
@Expose
protected String maskLength = SAME;

@SerializedName("truncateDigits")
@Expose
protected Integer truncateDigits;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think truncateLength is more descriptive? I noticed on line 393 you did refer to it as length in the error message.

When I first read it I interpreted it as "the specific digits you want to truncate" even though that doesn't make a lot of sense.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was mainly matching what the zip filter strategy already had, but I'm not sure I love either digits or length. I would have expected based on the name to have it truncate N digits rather than leaving N digits.

Maybe something like truncateLeaveCharacters or something would be more intuitive?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, yes, truncateDigits is what's in the zip code filter. I forgot about that. That used to make sense and now I don't like it.

I see your point. truncateLeaveCharacters makes sense. I think that's clearer. I'll write a new issue to make the property in ZipCodeFilterStrategy consistent with what you use here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed the parameter name to truncateLeaveCharacters in both the new code as well as the ZipCodeFilterStrategy. I made it backwards compatible so if truncateDigits is specified it will be used, but marked it as deprecated. I updated the docs to reference truncateLeaveCharacters.

@jzonthemtn
Copy link
Member

Looks great and definitely a valid use-case! Just one question about the property name.

@jzonthemtn jzonthemtn added this to the 2.10.0 milestone Dec 18, 2024
@jzonthemtn jzonthemtn merged commit 1ba9398 into philterd:main Dec 19, 2024
6 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants