[ic] ic-utf8 readfile/writefile patch

[Stefan Hornburg (Racke)]

Stefan Hornburg (Racke) wrote:
> David Christensen wrote:
>> Folks,
>>
>> I've added a patch to the ic-utf8 tree to support encoding/fallback  
>> strategy in Vend::File::readfile and writefile.  This is intended to  
>> be completely backwards-compatible with both legacy encodings and the  
>> current MV_UTF8 scheme while offering the following benefits:
>>
>>   - Explicit override of the encoding of any specific file.  This  
>> defaults to nothing (aka raw) when MV_UTF8 is not set, and utf-8 when  
>> MV_UTF8 is set.
>>   - Sensible default fallback to provide maximum information in the  
>> case that invalid encoding/decoding sequences are encountered.   
>> (Fallback strategy is how we deal with invalid/incomplete characters.)
>>   - Think future modifications to [include] to provide access to  
>> encoding and fallback parameters:  [include file="foo/bar/baz"  
>> encoding="cp1252"]
>>
>> I'd appreciate testing of this patch; in particular, this should help  
>> with Racke's issue encountered with legacy encodings on the index page  
>> with MV_UTF8 set.
> 
> Yes, it doesn't crash anymore :-).
> 
> One problem that might be related to this issue is the delivery of "binary" 
> content stored in a UTF8 database.
> 
> Currently the files produced are corrupted, and the data in the db is
> definitely correct. And it works with UTF8 inactive (as per bug #259).
> 
> The code is as follows:
> 
> my $data = $Db{transaction_documents}->field($td_code, 'content');
> $data = $Tag->filter({op => 'decode_base64', body => $data});

Putting at this point:

Encode::_utf8_on($data);

"solves" the problem.

> $Tag->deliver({type => 'application/pdf', 						body => $data});
> 

Regards
          Racke

-- 
LinuXia Systems => http://www.linuxia.de/
Expert Interchange Consulting and System Administration
ICDEVGROUP => http://www.icdevgroup.org/
Interchange Development Team