Item12569: Sorting in tableplugin is wrong in german
Priority: Normal
Current State: Closed
Released In: 2.1.0
Target Release: minor
If i sort a table with
TablePlugin in german umlaute are at the end of the table.
This looks like:
a
b
c
x
y
z
ä
ö
ü
but it should look like:
a
ä
b
c
ö
ü
x
y
z
--
FlorianPribahsnik - 23 Aug 2013
Same issue in our installation (Foswiki 1.1.8,
UseLocale=1, Site::Locale=de_AT.ISO-8859-1):
When using the column sort feature on a table like this
http://foswiki.org/Sandbox/GermanTableSort
the correct sort order would be A - Ä - B - ..., but instead the Umlauts show up at the very end of the table.
--
PhilippGortan - 24 Feb 2014
The patch seems to be rather trivial: When
UseLocale is enabled in the foswiki config, then the specified locale should be used for sorting. The implementation is identical to Foswiki::Search, Foswiki::Render, etc.
Item12569.patch: Proposed patch
--
PhilippGortan - 24 Feb 2014
Unfortunately
UseLocale
creates all sorts of other problems together with perl's tainted check.
--
MichaelDaum - 25 Feb 2014
The attached script
locale-test.pl tries to verify my assumption that:
- using "use locale" is scoped, so calling it in a block does not have any effect on the outer code
- "use locale" can thus be switched on and off at runtime without using the begin-if-require-import-hack
- outside of the block, \w does not taint the submatch
Can someone please verify that? Calling the script without params sorts without locale, adding any parameter when calling sorts with locale. In both cases, $res is not tainted.
--
PhilippGortan - 25 Feb 2014
Jan, Could you validate the addition of the
UseLocale patch to
TablePlugin/Core.pm? The proposed fix here is the same as what you used in just about every other module. Mark this waiting for release if it looks good. Thanks.
--
GeorgeClark - 12 Jan 2015
Definitely not fixed in 1.2.0. Note that client side sorting in javascript is also broken. In the core perl side, the fundamental issue is that we do not "normalize" the unicode characters. The following patch fixes the perl table sort implementation, but is NOT applied, as this needs much more discussion. Also the javascript side would need equivalent work.
diff --git a/TablePlugin/lib/Foswiki/Plugins/TablePlugin/Core.pm b/TablePlugin/lib/Foswiki/Plugins/TablePlugin/Core.pm
index 7764d25..9c9065a 100644
--- a/TablePlugin/lib/Foswiki/Plugins/TablePlugin/Core.pm
+++ b/TablePlugin/lib/Foswiki/Plugins/TablePlugin/Core.pm
@@ -10,6 +10,8 @@ use Foswiki::Plugins::TablePlugin ();
use Foswiki::Time;
use Error qw(:try);
+use Unicode::Normalize;
+
BEGIN {
if ( $Foswiki::cfg{UseLocale} ) {
require locale;
@@ -1502,7 +1504,7 @@ sub emitTable {
&& $currentSortDirection == $SORT_DIRECTION->{'ASCENDING'} )
{
@curTable = sort {
- $a->[$sortCol]->{sortText} cmp $b->[$sortCol]->{sortText}
+ NFKD($a->[$sortCol]->{sortText}) cmp NFKD($b->[$sortCol]->{sortText})
|| $a->[$sortCol]->{number} <=> $b->[$sortCol]->{number}
|| $a->[$sortCol]->{dateString}
cmp $b->[$sortCol]->{dateString}
@@ -1512,7 +1514,7 @@ sub emitTable {
&& $currentSortDirection == $SORT_DIRECTION->{'DESCENDING'} )
{
@curTable = sort {
- $b->[$sortCol]->{sortText} cmp $a->[$sortCol]->{sortText}
+ NFKD($b->[$sortCol]->{sortText}) cmp NFKD($a->[$sortCol]->{sortText})
|| $b->[$sortCol]->{number} <=> $a->[$sortCol]->{number}
|| $b->[$sortCol]->{dateString}
cmp $a->[$sortCol]->{dateString}
--
GeorgeClark - 25 Jun 2015
See discussion
In the IRC Logs
--
GeorgeClark - 25 Jun 2015
Fixed in
EditRowPlugin, so removed from the "Applies To"
--
Main.CrawfordCurrie - 28 Sep 2015 - 08:49