Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add the 'keep_tokens' parameter to 'filter_extremes' and test it #1210

Merged
merged 2 commits into from
Mar 13, 2017
Merged

Add the 'keep_tokens' parameter to 'filter_extremes' and test it #1210

merged 2 commits into from
Mar 13, 2017

Commits on Mar 13, 2017

  1. Add the 'keep_tokens' parameter to 'filter_extremes' and test it 

    Add the optional 'keep_tokens' parameter to the 'filter_extremes'
method in dictionary.py. This parameter can contain a list of tokens,
which will be kept regardless of the 'no_below' and 'no_above' settings.
This can be useful if the research goal is to enforce certain tokens to
appear in topics, and still be able to filter all other extremes.

If 'keep_tokens' is not given, the functionality of 'filter_extremes' is
unchanged.

Unit tests are also provided to assert examples of the above.
    @toltoxgh
    toltoxgh committed Mar 13, 2017
    Configuration menu
    Copy the full SHA
    9b4576a View commit details
    Browse the repository at this point in the history

  2. Create good_ids only once 

    Create good_ids only once as per optimization
suggestion, regardless if 'keep_tokens' is provided or not.
    @toltoxgh
    toltoxgh committed Mar 13, 2017
    Configuration menu
    Copy the full SHA
    ee6b4f7 View commit details
    Browse the repository at this point in the history