Ticket #2513 (assigned enhancement)

Opened 23 months ago

Last modified 20 months ago

Dataproxy should not default to utf8

Reported by: ross Owned by:
Priority: awaiting triage Milestone: ckan-backlog
Component: ckan Keywords:
Cc: Repository: ckan
Theme: none

Description

Unless explicitly told by the source web server the dataproxy should not assume that the content it has can be encoded as UTF-8. Even though the chars from 128 - 255 overlap an attempt to decode some byte array as utf8 will fail whenever a latin1 char whose bitpattern has the MSB set.

This will mean that the UTF8Recoder can be more rigid in its acceptance of data, Postel aside.

Change History

comment:1 Changed 23 months ago by ross

  • Milestone changed from ckan-v1.8 to ckan-v1.9

Have pushed back to 1.9 but if may be the the converter service will replace it in the meantime.

comment:2 Changed 20 months ago by ross

  • Owner ross deleted
  • Status changed from new to assigned
  • Milestone changed from ckan-v1.9 to ckan-backlog
Note: See TracTickets for help on using tickets.