Literal Whitespace in RegExps not properly parsed

[p5pRT]

Migrated from rt.perl.org#5051 (status was 'resolved')

Searchable as RT5051$

[p5pRT]

From Matthew.Persico@lazard.com

-----------------------------------------------------------------
The following script was run under Perl 5.004_04, 5.6.0 and
5.6.1 TRIAL 1

##Start

$x="20010103.09​:04​:10​: single line\n";

print "** with spaces​:\n";
if ($x=~/[0-9]{6}\.[0-2][0-9]​:([0-5][0-9]​:){2} single line\n/) {
  print "matched\n";
} else {
  print "no match? gee, this is really borken\n";
}

print "** with whitespace control char​:\n";
if ($x=~/[0-9]{6}\.[0-2][0-9]​:([0-5][0-9]​:){2}\ssingle\sline\n/) {
  print "matched\n";
} else {
  print "no match? gee, this is really borken\n";
}
##End

Under 5.6.0, the regexp is not matched when using literal whitespace
instead of whitespace control characters. This problem does not
occur in 5.004_04 or 5.6.1 trial 1​:

Results for your examination​:

5.004_04​:

Script started on Wed Jan 03 09​:12​:56 2001

persicom@​amdev01
Perl - 5.004_04
Pvcs - LF-CommonLog
/sa/common/src/perl/LF/CommonLog(8770) [409] $ perl regexp_test.pl
** with spaces​:
matched
** with whitespace control char​:
matched

5.6.0​:

Script started on Wed Jan 03 09​:12​:36 2001

persicom@​amdev01
Perl - 5.6.0
Pvcs - LF-CommonLog
/sa/common/src/perl/LF/CommonLog(8759) [405] $ perl regexp_test.pl
** with spaces​:
no match? gee, this is really borken
** with whitespace control char​:
matched

5.6.1​:

Script started on Wed Jan 03 09​:12​:07 2001

persicom@​amdev01
Perl - 5.6.1
Pvcs - none
/sa/common/src/perl/LF/CommonLog(8696) [401] $ perl regexp_test.pl
** with spaces​:
matched
** with whitespace control char​:
matched

Summary of my perl5 (revision 5.0 version 6 subversion 1) configuration​:
  Platform​:
  osname=solaris, osvers=2.6, archname=sun4-solaris
  uname='sunos amdev01 5.6 generic_105181-16 sun4u sparc sunw,ultra-enterprise
'
  config_args=''
  hint=previous, useposix=true, d_sigaction=define
  usethreads=undef use5005threads=undef useithreads=undef
usemultiplicity=undef
  useperlio=undef d_sfio=undef uselargefiles=define usesocks=undef
  use64bitint=undef use64bitall=undef uselongdouble=undef
  Compiler​:
  cc='cc', ccflags ='-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
  optimize='-O',
  cppflags='-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'
  ccversion='', gccversion='', gccosandvers=''
  intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=4321
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
  ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
  alignbytes=8, usemymalloc=y, prototype=define
  Linker and Libraries​:
  ld='cc', ldflags =' -L/usr/local/lib -L/opt/local/lib
-L/opt/SUNWspro/SC4.2/lib '
  libpth=/usr/local/lib /opt/local/lib /opt/SUNWspro/SC4.2/lib /lib /usr/lib
/usr/ccs/lib
  libs=-lsocket -lnsl -ldl -lm -lc -lcrypt -lsec
  perllibs=-lsocket -lnsl -ldl -lm -lc -lcrypt -lsec
  libc=/lib/libc.so, so=so, useshrplib=false, libperl=libperl.a
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags=' '
  cccdlflags='-KPIC', lddlflags='-G -L/usr/local/lib -L/opt/local/lib
-L/opt/SUNWspro/SC4.2/lib'

Characteristics of this binary (from libperl)​:
  Compile-time options​: USE_LARGE_FILES
  Locally applied patches​:
  v5.6.1-TRIAL1
  Built under solaris
  Compiled at Dec 22 2000 12​:55​:25
  %ENV​:
  PERL5LIB="/sa/common/lib/site_perl/5.6.1​:/am/common/lib/site_perl/5.6.1"
  PERL5MAN="/am/perl/man/5.6.1"
  @​INC​:
  /sa/common/lib/site_perl/5.6.1
  /am/common/lib/site_perl/5.6.1
  /am/perl/lib/5.6.1/sun4-solaris
  /am/perl/lib/5.6.1
  /am/perl/lib/site_perl/5.6.1/sun4-solaris
  /am/perl/lib/site_perl/5.6.1
  /am/perl/lib/site_perl
  .
script done on Wed Jan 03 09​:12​:23 2001

Furthermore, look what happens with these two scripts that
attempt to match across lines (/s qualifier) under 5.6.0. You'll
see why I had to break it up​:

$x="multi line...20010103.09​:46​:25​: multi
  : line
";
print "** with whitespace control char​:\n";
if ($x=~/[0-9]{6}\.[0-2][0-9]​:([0-5][0-9]​:){2}\smulti\n {17}​: line\n/s) {
  print "matched\n";
} else {
  print "no match? Gee, this is borken!\n";
}

persicom@​amdev01
Perl - 5.6.0
Pvcs - LF-CommonLog
/sa/common/src/perl/LF/CommonLog(24067) [436] $ perl regexp_test.pl
** with whitespace control char​:
matched

$x="multi line...20010103.09​:46​:25​: multi
  : line
";
print "** with spaces​:\n";
if ($x=~/[0-9]{6}\.[0-2][0-9]​:([0-5][0-9]​:){2} multi\n {17}​: line\n/s) {
  print "matched\n";
} else {
  print "no match? Gee, this is borken!\n";
}

persicom@​amdev01
Perl - 5.6.0
Pvcs - LF-CommonLog
/sa/common/src/perl/LF/CommonLog(24067) [436] $ perl regexp_test.pl
Out of memory during ridiculously large request at p2 line 6.

Note that there is whitespace before the {17} and the 'line\n' text that works
fine.
It looks like the whitespace that follows a {} count specifier may be the
problem.

Perl Info

Flags:
    category=core
    severity=medium

Site configuration information for perl v5.6.0:

Configured by persicom at Fri Sep  8 22:11:29 EDT 2000.

Summary of my perl5 (revision 5.0 version 6 subversion 0) configuration:
  Platform:
    osname=solaris, osvers=2.6, archname=sun4-solaris
    uname='sunos amdev01 5.6 generic_105181-16 sun4u sparc sunw,ultra-enterprise
'
    config_args=''
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=undef use5005threads=undef useithreads=undef
usemultiplicity=undef
    useperlio=undef d_sfio=undef uselargefiles=define
    use64bitint=undef use64bitall=undef uselongdouble=undef usesocks=undef
  Compiler:
    cc='cc', optimize='-O', gccversion=
    cppflags=''
    ccflags =' -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'
    stdchar='unsigned char', d_stdstdio=define, usevfork=false
    intsize=4, longsize=4, ptrsize=4, doublesize=8
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
    alignbytes=8, usemymalloc=y, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/local/lib -L/opt/local/lib
-L/opt/SUNWspro/SC4.2/lib '
    libpth=/usr/local/lib /opt/local/lib /opt/SUNWspro/SC4.2/lib /lib /usr/lib
/usr/ccs/lib
    libs=-lsocket -lnsl -ldl -lm -lc -lcrypt -lsec
    libc=/lib/libc.so, so=so, useshrplib=false, libperl=libperl.a
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags=' '
    cccdlflags='-KPIC', lddlflags='-G -L/usr/local/lib -L/opt/local/lib
-L/opt/SUNWspro/SC4.2/lib'

Locally applied patches:



@INC for perl v5.6.0:
    /home/persicom/local/lib/perl/5.6.0/lib/5.6.0
    /home/persicom/local/lib/perl/5.6.0/lib/site_perl
    /sa/common/lib/site_perl/5.6.0
    /am/common/lib/site_perl/5.6.0/sun4-solaris
    /am/common/lib/site_perl/5.6.0
    /am/perl/lib/5.6.0/sun4-solaris
    /am/perl/lib/5.6.0
    /am/perl/lib/site_perl/5.6.0/sun4-solaris
    /am/perl/lib/site_perl/5.6.0
    /am/perl/lib/site_perl
    .


Environment for perl v5.6.0:
    HOME=/home/persicom
    LANG (unset)
    LANGUAGE (unset)
    LD_LIBRARY_PATH=/am/common/lib:/opt/documentum/4.0.5/lib:/opt/sybase/lib
    LOGDIR (unset)

PATH=/home/persicom/local/lib/perl/5.6.0/bin:/am/common/bin/5.6.0:/sa/common/bin/5.6.0:/am/perl/bin/5.6.0:/home/persicom/local/bin:/opt/local_dev/xemacs-21.2.35/bin:/opt/local_dev/bin:/home/amdev/pvcs:/am/common/bin:/am/portia.50i/apotest02:/sa/common/bin:/opt/oracle/bin:/opt/documentum/4.0.5/bin:/opt/sybase/bin:/opt/Acrobat3/bin:/opt/autosys/bin:/usr/java1.2/bin:/usr/ccs/bin:/usr/openwin/bin:/opt/SUNWspro/bin:/usr/local/bin:/usr/bin

PERL5LIB=/home/persicom/local/lib/perl/5.6.0/lib/5.6.0:/home/persicom/local/lib/perl/5.6.0/lib/site_perl:/sa/common/lib/site_perl/5.6.0:/am/common/lib/site_perl/5.6.0
    PERL5MAN=/am/perl/man/5.6.0
    PERL_BADLANG (unset)

[p5pRT]

From @floatingatoll

[Matthew.Persico@​lazard.com - Tue Jan 2 23​:01​:11 2001]​:

The following script was run under Perl 5.004_04, 5.6.0 and
5.6.1 TRIAL 1

##Start

$x="20010103.09​:04​:10​: single line\n";

print "** with spaces​:\n";
if ($x=~/[0-9]{6}\.[0-2][0-9]​:([0-5][0-9]​:){2} single line\n/) {
print "matched\n";
} else {
print "no match? gee, this is really borken\n";
}

print "** with whitespace control char​:\n";
if ($x=~/[0-9]{6}\.[0-2][0-9]​:([0-5][0-9]​:){2}\ssingle\sline\n/) {
print "matched\n";
} else {
print "no match? gee, this is really borken\n";
}
##End

This test program passes, as of 5.8/5.9 devel.

$x="multi line...20010103.09​:46​:25​: multi
: line
";
print "** with whitespace control char​:\n";
if ($x=~/[0-9]{6}\.[0-2][0-9]​:([0-5][0-9]​:){2}\smulti\n {17}​:
line\n/s) {
print "matched\n";
} else {
print "no match? Gee, this is borken!\n";
}

This test, however, does not pass, as of @​17821.

- R.

[p5pRT]

From @demerphq

This bugs appears to be resolved in later versions of perl. In a recent
version of bleadperl​:

D​:\dev\perl\ver\28461_>.\perl c​:\tmp\more_re_bugs.pl
** with spaces​:
matched
** with whitespace control char​:
matched

Same result in perl 5.8.6

Yves

[p5pRT]

@demerphq - Status changed from 'open' to 'resolved'