Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle pycld2 errors #1

Closed
leezu opened this issue Jan 29, 2018 · 3 comments
Closed

Handle pycld2 errors #1

leezu opened this issue Jan 29, 2018 · 3 comments

Comments

@leezu
Copy link
Contributor

leezu commented Jan 29, 2018

Thanks @nickdavidhaynes for writing this wrapper.

I believe it should handle errors thrown by pycld2. Currently the whole spacy pipeline may crash, if pycld2 throws an error such as pycld2.error: input contains invalid UTF-8 around byte 977 (of 4604). Instead it may be better to return an empty language tuple (as is done if the language is unknown). What do you think?

@nickdavidhaynes
Copy link
Owner

Agreed, error handling is pretty pitiful at the moment.

Do you happen to have a string that throws the error you see there?

@leezu
Copy link
Contributor Author

leezu commented Jan 30, 2018

I changed your code to print the document on error (and to then ignore the error, classifying the sentence as "error"). But I got the following document, which parses fine when checking manually with cld.

In the interest of time and space I will list 12 examples of how Jen has been amazing with my animals.
1.  My husband and I had a trip planned and were using a kennel for our young dog, as Jen had been booked for the overnights and could not stay with our dog, but could stop in for 3 visits a day for my cats.  At the time, I had only one special needs cat that required someone to feed her 3 times a day.  Jen had that covered. However, the kennel where I was boarding my dog was not sure they had space for my dog for the duration of our stay.  Jen was going to go pick her up and take her to her house with her dogs.  She stopped by my house and got the extra crate and food.  As it turned out, the kennel made arrangements for my dog for the entire stay, but the fact Jen was will to go above and beyond to make our trip seamless, was quite impressive.
2.  On another visit, I was away at the beach with a girlfriend, my husband was home with the pets but still had to go �to work, so Jen was coming in and doing feedings and scoop kitty litter while my husband was at work.  I had a cat that needed a follow-up visit to the vet and I had asked Jen to do it rather than have my husband do it, as I knew Jen would ask all the right follow-up questions and how to proceed going forward.  While loading my cat into the car, she noticed my indoor/outdoor cat and noticed he was limping/bleeding.  She immediately scooped him up and took him with her to the vet. Thank goodness she did.  He had lost a toenail and it had become infected.  Jen is just amazing that way.
3.  I was training my dog to be a therapy dog and was not happy with one of the companies I was given an option to test with.  Jen not only introduced me to the group she was working with, but went with me to the meetings and first visit.
4.  One night at around 10:30 p.m. I was trimming the back nails of one of my cats.  I had cut too much and the toenail started to bleed. I had none of the antiseptic powder that stops the bleeding.  Immediately I texted Jen even though it was so late.  She told me what alternative products I could use and offered to bring over her antiseptic powder.  One of the alternatives worked.
5.  I came home from a vacation to a sign that read "Caution, Cat Crossing" as I have an indoor/outdoor cat that will sleep in the driveway or dart out of a shrub as you drive around my driveway to the garage.  Now, everyone who comes up my driveway slows down for my baby boy.  Thanks Jen!!!
6.  My mother adopted a 9-year-old rescue dog.  It was having trouble eating and was constantly itching.  I told Jen, she immediately started texting and facebooking with my my to help her find a solution.  Th�ey did.  With Jen's help, they figured out the dog had a food allergy and now my mom's dog is healthy and happy.  Any issues my mother has, she goes directly to Jen.
7.  I had done formal training sessions as well as private lessons with my dog, Darby, from the time she was 12 weeks old.  She is now 3.5 years old.  I could not EVER get her to "give paw."  I finally resigned myself that my dog was never going to be a dog that gives paw.  I came home from a trip with my husband and Jen said, "show mommy what you can do."  Darby gave paw.  UNBELIEVABLE!!!  In addition, Darby will do a sitting "Leave it."  While sitting with her, a treat is dropped in front of her and she is told to leave it until commanded to do otherwise.  I had been unable to get her to do it, but Jen had her master it.  
8.  Jen is very good at texting after each visit with a brief update.
9.  In all the years of using Jen, she has never canceled, been late or forgotten a single visit.
10.  She will schedule play dates with neighborhood dogs when watching multiple neighbor dogs.  They all get along so great.  
11.  Most recently, my 2nd special needs kitty was having issues and the vet recommended we have her scoped.  I was not ready to do that, but I contacted Jen immediately to get her thoughts and advice.  She made a bunch of calls on my behalf to check on holistic treating vets.  Jen and I are both willing to do holistic treatments on our animals.  Jen brought over an herb mixture for my baby, and she has been doing great ever since.  
12.  I have a cat with no teeth and is a very picky eater.  She will only eat roaster chicken, shredded turkey lunch meat and human tuna.  While away on trips, Jen will stop at the grocery store and pick up fresh food for my kitty as I cannot provide enough for a trip lasting longer than 3-4 days.
I could go on and on, but there is nothing she will not do for her clients.

This is the source of the document

For some reason it includes the Unicode Character 'DELETE' (U+007F) at position 977.

leezu added a commit to leezu/spacy-cld that referenced this issue Jan 30, 2018
@leezu
Copy link
Contributor Author

leezu commented Jan 30, 2018

The exact parsing problem should probably be fixed in the pycld2 library. Nevertheless I sent you a Pull Request so that spacy-cld won't crash the whole pipeline due to issues in pycld2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants