Import: modifier „Regular expression based replacement" writes non-matched string to capture-group

[tectumopticum]

Expected Behavior

During import the modifier „Regular expression based replacement" should extract a part of a string and write it - if it matched correctly - by using the regex-capture-variable "$1" into a new field (Target property). Otherwise a capture-group ("$1") should be empty or NULL.

Current Behavior

If the regex doesn't match the whole string is written into the new field instead of NULL/empty string.

Steps to Reproduce (for bugs)

Create import-source with modifier „Regular expression based replacement". Example for
"Regex pattern":
/^[a-z]{2}[a-z0-9]{3}([a-z]+)[0-9]+.*$/
"Replacement":
$1

matching-cases:
abcdexxxx01 => $1: xxxx
efgh1yyy10 => $1: yyy

non-matching-cases
abcde31yyy01 => $1: abcde31yyy01
abcde2yyy01 => $1: abcde2yyy01
abcdexxx-yyy => $1: abcdexxx-yyy

Your Environment

• Director version (System - About): 1.10.2
• Icinga Web 2 version and modules (System - About): 2.11.3
• Icinga 2 version (icinga2 --version): 2.13.6
• Operating System and version: SLES 15 SP3
• Webserver, PHP versions: Apache/2.4.51 (Linux/SUSE), PHP 7.4.33 (cli) ( NTS )

[MisterMountain]

ref/NC/776129

[Thomas-Gelf]

This is correct and expected. A replacement based on regular expressions replaces parts of your string that match a given pattern with the given replacement. No match, no replacement.

[tectumopticum]

Sorry, but I cannot agree to this because it's imho contrary to the common regex-behaviour in other applications or languages or (training-)documentation. If you setup a capture-group e.g. in PERL and your string doesn't match, the variables remain empty and are not filled with your string or parts of it.
As a reference I always check this with the well-known "regex-testers" (e.g. https://regex101.com/).
Furthermore I wanted to write the output into a new field and not replace a string in the given one. What could be a more suitable alternative?

[Thomas-Gelf]

That's correct, the variables remain empty. That's what happens in Director too. But what also happens is, that the provided string will NOT be modified - like in every other application, even in Perl.

What you're running corresponds to the following Perl snippet:

#!/usr/bin/perl

foreach (
  "abcdexxxx01",
  "efgh1yyy10",
  "abcde31yyy01",
  "abcde2yyy01",
  "abcdexxx-yyy"
) {
  my $host = $_;
  print "$host = ";
  $host =~ s/^[a-z]{2}[a-z0-9]{3}([a-z]+)[0-9]+.*$/$1/;
  print "$host\n";
}

It shows the following output:

abcdexxxx01 = xxxx
efgh1yyy10 = yyy
abcde31yyy01 = abcde31yyy01
abcde2yyy01 = abcde2yyy01
abcdexxx-yyy = abcdexxx-yyy

To understand why your requirement would cause problems, try to imagine what would happen with a pattern NOT matching the full string, not ranging from ^ to $. In your believes, how should Director behave for the pattern /xxx/ with the replacement 'aaa'?

[Thomas-Gelf]

Said all this, I already have an idea of how to help you with your requirement ;-)

[Thomas-Gelf]

Hi @tectumopticum,

I enhanced the modifier with a new configuration flag. It's default behavior remains as-is:

But there is now the possibility to tweak this:

Please apply the following patch and let me know, whether this helps to address your requirement.

diff --git a/library/Director/PropertyModifier/PropertyModifierRegexReplace.php b/library/Director/PropertyModifier/PropertyModifierRegexReplace.php
index 59cb245e..fdf82ae9 100644
--- a/library/Director/PropertyModifier/PropertyModifierRegexReplace.php
+++ b/library/Director/PropertyModifier/PropertyModifierRegexReplace.php
@@ -23,6 +23,17 @@ class PropertyModifierRegexReplace extends PropertyModifierHook
                 'The string that should be used as a preplacement'
             ),
         ));
+        $form->addElement('select', 'when_not_matched', [
+            'label'       => $form->translate('When not matched'),
+            'description' => $form->translate(
+                "What should happen, if the given pattern doesn't match"
+            ),
+            'value' => 'keep',
+            'multiOptions' => [
+                'keep'     => $form->translate('Keep the given string'),
+                'set_null' => $form->translate('Set the value to NULL')
+            ]
+        ]);
     }
 
     public function getName()
@@ -36,10 +47,17 @@ class PropertyModifierRegexReplace extends PropertyModifierHook
             return null;
         }
 
-        return preg_replace(
+        $result = preg_replace(
             $this->getSetting('pattern'),
             $this->getSetting('replacement'),
             $value
         );
+        if ($result === $value && $this->getSetting('when_not_matched', 'keep') === 'set_null') {
+            if (!preg_match($this->getSetting('pattern'), $value)) {
+                return null;
+            }
+        }
+
+        return $result;
     }
 }

In my lab, the result changes from:

...to:

As I didn't want to always call preg_match before preg_replace, the extra logic is now only applied, when the output string matches the input string after running the replacement operation.

[tectumopticum]

Hey @Thomas-Gelf ,
perfect! Tested and works as expected. Now it's easier to react to something which doesn't match e.g. the defined naming-conventions.
Thanks a lot for your efforts & best regards!

[Thomas-Gelf]

@tectumopticum: glad to hear that, you're welcome. This has been merged and will be released with v1.11.0.