[ic] Filters with UTF-8 body

[David Christensen]

On Sep 14, 2009, at 8:51 AM, Stefan Hornburg (Racke) wrote:

> David Christensen wrote:
>> On Mar 12, 2009, at 2:10 PM, Peter wrote:
>>
>>> On 03/12/2009 06:12 AM, David Christensen wrote:
>>>> On Mar 12, 2009, at 5:31 AM, Peter wrote:
>>>>
>>>>> On 03/12/2009 03:17 AM, Peter wrote:
>>>>>> On 03/12/2009 03:04 AM, Stefan Hornburg wrote:
>>>>>>> Peter Ajamian suggested that the following code in  
>>>>>>> Interpolate.pm
>>>>>>> causes the problem:
>>>>>>>
>>>>>>> '_filter'               => qr($T{_filter}\s+($Some)\]($Some)),
>>>>>>> my $Some = '[\000-\377]*?';
>>>>>> More specifically $Some, $All, $XSome and $XAll will only parse 8
>>>>>> bit
>>>>>> characters in the range \000-\377.  Not positive about this,  
>>>>>> but I
>>>>>> think
>>>>>> that changing them to the following will work:
>>>>>> my $All = '(?:(?s).*)';
>>>>>> my $Some = '(?:(?s).*?)';
>>>>>> my $XAll = qr{(?:(?s).*)};
>>>>>> my $XSome = qr{(?:(?s).*?)};
>>>>> On further reflection this would probably work just as well and is
>>>>> less
>>>>> complex looking:
>>>>> my $All = '[.\n]*';
>>>>> my $Some = '[.\n]*?';
>>>>> my $XAll = qr{[.\n]*};
>>>>> my $XSome = qr{[.\n]*?};
>>>>
>>>> Heh, one problem:
>>>>
>>>> $ perl -e 'print "matches!" if "foo" =~ /[.\n]/'
>>>> $ perl -e 'print "matches!" if "foo" =~ /(.|[\n])/'
>>>> matches!
>>> Strange.  So may as well go with the (?s) solution as Jon says.  To
>>> add
>>> a few more tests:
>>> peter at peter-desktop:~$ perl -le 'print $1 if "foo\nbar" =~ /((?:(?
>>> s).*))/'
>>> foo
>>> bar
>>> peter at peter-desktop:~$ perl -le 'print $1 if "foo\nbar" =~ /((?:.|
>>> \n)*)/'
>>> foo
>>> bar
>>> peter at peter-desktop:~$ perl -le 'print $1 if "foo\nbar" =~ /(.*)/'
>>> foo
>>> peter at peter-desktop:~$
>>
>>
>> Okay, pushed commit to interchange-utf8; this is a patch that could  
>> be
>> applied to CVS separately if you guys want.  Racke, can you test this
>> against your issue?
>>
>
> David, it seems to me that this patch never went back in our main
> repository. Can you send the patch again?

This is now committed; I did some review to see if there were  
additional similar issues introduced since then, and there were not.

Regards,

David
--
David Christensen
End Point Corporation
david at endpoint.com