Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Manifest refactor #284

Merged
merged 6 commits into from
Jan 23, 2020
Merged

Manifest refactor #284

merged 6 commits into from
Jan 23, 2020

Conversation

stasbel
Copy link
Contributor

@stasbel stasbel commented Jan 22, 2020

Old usage:

manifest = ManifestBase(  # Or `ManifestEN`.
    *args,
    **kwargs,
)

New usage:

# More conventional full path imports with namespaces to import from.
from nemo_asr.parts import collections
from nemo_asr.parts import parsers


# Make parser.
parser = parsers.make_parser(labels, 'en', **kwargs)
parser = parsers.ENCharParser(labels, **kwargs)  # Or.

# Make collection.
collection = collections.Text(texts, parser=parser)
collection = collections.FromFileText(file, parser=parser)  # Or.

# Use fields. Each example is namedtuple.
# Look for `collections.Class.OUTPUT_TYPE` for fields information.
print(collection[0].tokens)   # List of ints.

@stasbel stasbel requested review from blisc and okuchaiev January 22, 2020 00:57
@lgtm-com
Copy link

lgtm-com bot commented Jan 22, 2020

This pull request introduces 1 alert and fixes 3 when merging 00a9d59 into bc0d7b1 - view on LGTM.com

new alerts:

  • 1 for Unused import

fixed alerts:

  • 3 for Unused import

collections/nemo_asr/nemo_asr/parts/dataset.py Outdated Show resolved Hide resolved
requirements.txt Show resolved Hide resolved
collections/nemo_asr/nemo_asr/parts/asr_manifest.py Outdated Show resolved Hide resolved
collections/nemo_asr/nemo_asr/parts/manifest.py Outdated Show resolved Hide resolved
scripts/decode.py Outdated Show resolved Hide resolved
@blisc
Copy link
Collaborator

blisc commented Jan 22, 2020

Why did you request a review for a WIP PR?

@blisc
Copy link
Collaborator

blisc commented Jan 22, 2020

The new classes inside manifests.py are not very intuitive to me. It's hard for me to understand what you are trying to do with them at first glance.

@blisc blisc removed the request for review from okuchaiev January 22, 2020 01:49
@stasbel
Copy link
Contributor Author

stasbel commented Jan 22, 2020

Why did you request a review for a WIP PR?

So you can comment on draft. Now I know what's need to fixed first.

@stasbel
Copy link
Contributor Author

stasbel commented Jan 22, 2020

Sorry, I have to force-push commits to re-sign them after rebasing.

@stasbel stasbel changed the title [WIP] Manifest refactor Manifest refactor Jan 22, 2020
@lgtm-com
Copy link

lgtm-com bot commented Jan 22, 2020

This pull request fixes 4 alerts when merging 8a22b19 into 7779728 - view on LGTM.com

fixed alerts:

  • 3 for Unused import
  • 1 for Except block handles 'BaseException'

@stasbel stasbel requested review from blisc and okuchaiev January 22, 2020 21:04
@stasbel stasbel requested a review from blisc January 22, 2020 22:10
@lgtm-com
Copy link

lgtm-com bot commented Jan 22, 2020

This pull request introduces 1 alert and fixes 4 when merging 59ba6c9 into 7779728 - view on LGTM.com

new alerts:

  • 1 for Unused import

fixed alerts:

  • 3 for Unused import
  • 1 for Except block handles 'BaseException'

Copy link
Collaborator

@blisc blisc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just import orders

collections/nemo_asr/nemo_asr/data_layer.py Show resolved Hide resolved
collections/nemo_asr/nemo_asr/parts/dataset.py Outdated Show resolved Hide resolved
collections/nemo_tts/nemo_tts/parts/datasets.py Outdated Show resolved Hide resolved
@lgtm-com
Copy link

lgtm-com bot commented Jan 22, 2020

This pull request fixes 4 alerts when merging 37d99b2 into 7779728 - view on LGTM.com

fixed alerts:

  • 3 for Unused import
  • 1 for Except block handles 'BaseException'

@stasbel stasbel requested a review from blisc January 22, 2020 22:52
@lgtm-com
Copy link

lgtm-com bot commented Jan 22, 2020

This pull request fixes 4 alerts when merging 410dbba into 7779728 - view on LGTM.com

fixed alerts:

  • 3 for Unused import
  • 1 for Except block handles 'BaseException'

Signed-off-by: Stanislav Beliaev <stasbelyaev96@gmail.com>
Signed-off-by: Stanislav Beliaev <stasbelyaev96@gmail.com>
Signed-off-by: Stanislav Beliaev <stasbelyaev96@gmail.com>
Signed-off-by: Stanislav Beliaev <stasbelyaev96@gmail.com>
Signed-off-by: Stanislav Beliaev <stasbelyaev96@gmail.com>
Signed-off-by: Stanislav Beliaev <stasbelyaev96@gmail.com>
@lgtm-com
Copy link

lgtm-com bot commented Jan 22, 2020

This pull request fixes 4 alerts when merging 5c9b906 into 30db6f2 - view on LGTM.com

fixed alerts:

  • 3 for Unused import
  • 1 for Except block handles 'BaseException'

@stasbel stasbel merged commit b986859 into master Jan 23, 2020
@stasbel stasbel deleted the manifest-refactor branch January 23, 2020 00:12
@blisc blisc mentioned this pull request Jan 25, 2020
dcurran90 pushed a commit to dcurran90/NeMo that referenced this pull request Oct 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants