unicodeTestIn script can be slow

[mauritsvanrees]

The unicodeTestIn script can be slow. The last change to this script was in 2010 by me. This speeded it up already, but today I saw that it can still be slow.

Case I saw today:

• Plone 4.3.11
• Over 3500 keywords/tags/subjects in the site.
• Edit a page that has no tags: takes about three seconds.
• Add five tags to the page.
• Edit the page again: takes about twelve seconds.

The tags are rendered by the keyword.pt widget template, which calls unicodeTestIn in a few lines. The culprit is this line in the rendering of the select tag and its options, which gets called often. Translated to Python code, that part of the template does this:

# currently selected tags:
value = ['a', 'b, 'c']
# allowedKeywords is bug list of 3500 keywords
for keyword in allowedKeywords:
    context.unicodeTestIn(keyword, value)

So unicodeTestIn is called 3500 times, and each time it calls context.unicodeEncode(v) on each item in the current value. The worst thing about that, is that unicodeEncode is a separate skin script, so Plone has to locate this script, check permissions, etcetera, so that is probably what makes it take a lot of time.

I have prepared a fix that brings the edit-page rendering in the example back to three seconds. I will create a PR shortly.
Should probably be done on all current Archetypes branches, but let's start with branch 1.9.x for Plone 4.3.