Payee auto renaming

[chaserhkj]

Hi there,

Is it possible to implement custom regex-based payee auto renaming before using the payee names to guess the account names? This is because some of my transactions fetched through OFX from Chase actually have a unique number attaches to the payee line and this makes the account name guessing unusable. If I could unify these by regex-replacing the payee lines first before doing the guessing this would be a lot more easier and tidier.

[egh]

There used to be some fuzzy matching for payees that might have helped, but it wasn't being used properly as far as I know. It might be possible to implement something regex based - I don't use the payee matching myself, so I'm not sure.

[hollyhudson]

There doesn't seem to be any matching of payees, unless I'm missing something? None of mine are being matched, they're all getting Expenses:Misc. I'd love a fix for this since this project is top-notch at downloading and processing the ofx file (seriously, excellent job!), but if the payees aren't matched it's not really usable for me due to the volume of transactions I'm dealing with, hand-editing would be horrible. Although since one of my banks only offers ofx (no csv) I might not have a choice but to try to figure out how to add the functionality, with my limited programming skills. :/

Hmm.. just to be clear, I'm talking about using the payee assertions in the journal file to choose the account for the other side of each transaction, which is what I assume is being discussed here, but I could be wrong.

[egh]

Thanks! I'm glad it's working well for you :) I realized I've never describe the payee matching that does happen, so I added some documentation to the README here: https://github.com/egh/ledger-autosync#payee-matching. I think payee matching is the weakest part of ledger-autosync, but I'm not sure where to go next with it.

[qypea]

Something I've done in my custom CsvConverter is provide 'shorter matching payee' support. So for instance the first time I see a transaction import with "AMZN Mktp US*MB2ED6XZ0 Amzn.com/billWA" I manually change the payee in my ledger to "AMZN Mktp US". Then any future imports which contain that string are automatically matched to the same account.

    def get_payee(self, row):
        self.lgr.load_payees()
        payee = row['Description']
        for existing in self.lgr.payees:
            # Search for a payee which is a substring, use it instead of full description
            if existing and existing in payee:
                payee = existing
                break

        return payee

Its not perfect by any means, but it does allow me to not have to fix the expense accounts nearly as often.

[hollyhudson]

Hm. The problem is, my first import into the ledger ecosystem is going to be 6 months worth of transactions -- I definitely do not want to categorize all them by hand. ledger convert supports matching on the payee field, and it worked perfectly for the few accounts I tried (the ones that don't offer ofx downloads) so I created a really long schema.ledger file with payee matching regular expressions to deal with categorization for me. Here's an excerpt from the file as an example of what I'm talking about:

account Expenses:Subscriptions
    payee ^NYC Resistor LLC
    payee ^Spotify
    payee ^GitHub
    payee ^Netflix
    payee ^The New York Times
    payee ^Lesson Nine

So, matching via that system is what I'm looking for.

[egh]

I didn't know that was a thing. Interesting.

Is there a way that you know to get ledger to report on that, e.g. to run ledger and output the data? ledger-autosync doesn't have a way to parse ledger files, only the output from ledger/hledger. I'd rather not get into the business of parsing ledger files, even as simple a construct as that, because I don't want to have to maintain compatibility. It looks like the xact command might be able to help use ledger's account guessing mechanism, but I can't figure out how to make it output something.

[hollyhudson]

Yeah, I peaked at convert.cc in the ledger source code to try to see how they match things internally for clues on how to do it externally, but c++ code that isn't blinking a light or reading from a sensor is kind of opaque to me :/

But for parsing the ledger files in general.. is that not what the -l flag for autosync does? I give it my main journal file, that includes that account schema along with all the other files generated by ledger-autosync, ledger convert, and the few I write by hand, so the account directives with their payee fields should be available to ledger-autosync (but yeah, not sure how to process that data once you have it).

[qypea]

One of the ?easier? ways I'm finding to have ledger do this conversion for you is to call ledger print. We may be able to do this as a finishing up step after creating the transaction, or I'm wondering if I actually may want to leave all of my transactions marked with "Expenses:Imported" or a similar generic account and then map payees to accounts dynamically every time I run a query on my ledger file.

Sample ledger.ldg file:

account Expenses:Food
    payee ^Bobs Diner

2019/06/20 * Bobs Diner 5001
    ; csvid: 000101
    Assets:WellsFargo:Checking                            -$12.00
    Expenses:Unknown

Output of ledger print -E meta csvid=000101:

2019/06/20 * Foobar
    ; csvid: 000101
    Assets:WellsFargo:Checking               $-12.00
    Expenses:Food

[egh]

@qypea Thank you! That seems like something that we could reuse to leverage ledger's payee matching.

[dtgnn]

Hello,

I am a new user of ledger and ledger-autosync. I came here looking for info about a related issue and thought I might ask my question here.

I am trying to make use of the AutosyncPayee tag, as documented in the README. But I haven't been successful. Payee matching works just fine if I don't mess with the payee information and the correct account is guessed from previous transactions. Everything breaks, though, if I try to shorten the payee description and move the original one in the commends, after the AutotagPayee: tag.

I am using ledger-autosync v1.0.1 installed through pip.

I realize the original issue discussed is more complicated than this one, so sorry if this is not the place for this question -- didn't mean to hijack the thread. Let me know if I should post in a new one.

And thank you for developing this useful tool!

[egh]

I don't in fact use the payee matching, so I'm not sure what the issue could be. @501st-alpha1 implemented that & might have some information.

[501st-alpha1]

@dtgnn Could you please post a (minimal) example showing what doesn't work? I agree that this sounds like a different problem, so feel free to open a new issue and we can continue the conversation there.