[iOS] Calendar not behaving correctly wrt culture #50859

spouliot · 2021-04-07T18:17:06Z

Description

I am re-enabling some tests to confirm ICU is working correctly to replace the legacy mono code for iOS.

Most globalization tests have been re-enabled successfully, except for Calendar tests

https://github.com/xamarin/xamarin-macios/blob/main/tests/linker/ios/dont%20link/CalendarTest.cs

Those are not returning the expected types.

DontLink

DontLink.Calendars

DontLink.Calendars.CalendarTest
	[FAIL] Hijri :   Calendar
  Expected string length 34 but was 38. Strings differ at index 21.
  Expected: "System.Globalization.HijriCalendar"
  But was:  "System.Globalization.GregorianCalendar"
  --------------------------------^

		   at DontLink.Calendars.CalendarTest.Hijri() in /Users/poupou/git/main/xamarin-macios/tests/linker/ios/dont link/CalendarTest.cs:line 28
	[FAIL] ThaiBuddhist :   Calendar
  Expected string length 41 but was 38. Strings differ at index 21.
  Expected: "System.Globalization.ThaiBuddhistCalendar"
  But was:  "System.Globalization.GregorianCalendar"
  --------------------------------^

		   at DontLink.Calendars.CalendarTest.ThaiBuddhist() in /Users/poupou/git/main/xamarin-macios/tests/linker/ios/dont link/CalendarTest.cs:line 35
	[FAIL] UmAlQura :   Calendar
  Expected string length 37 but was 38. Strings differ at index 21.
  Expected: "System.Globalization.UmAlQuraCalendar"
  But was:  "System.Globalization.GregorianCalendar"
  --------------------------------^

		   at DontLink.Calendars.CalendarTest.UmAlQura() in /Users/poupou/git/main/xamarin-macios/tests/linker/ios/dont link/CalendarTest.cs:line 21
DontLink.Calendars.CalendarTest : 226.39 ms
DontLink.Calendars : 229.61 ms

Configuration

dotnet SDK 6.0.100-preview.3.21202.5
the ICU enabled code is not yet merged - in part since I can't re-enable this test
unit test Dont link (where the linker is not used) shows the issue

Regression?

Yes. This is working on the current Xamarin.iOS SDK.

Other information

xamarin-macios tracking issue for globalization

The text was updated successfully, but these errors were encountered:

ghost · 2021-04-07T18:17:11Z

Tagging subscribers to this area: @tarekgh, @safern
See info in area-owners.md if you want to be subscribed.

Issue Details

Description

I am re-enabling some tests to confirm ICU is working correctly to replace the legacy mono code for iOS.

Most globalization tests have been re-enabled successfully, except for Calendar tests

https://github.com/xamarin/xamarin-macios/blob/main/tests/linker/ios/dont%20link/CalendarTest.cs

Those are not returning the expected types.

DontLink

DontLink.Calendars

DontLink.Calendars.CalendarTest
	[FAIL] Hijri :   Calendar
  Expected string length 34 but was 38. Strings differ at index 21.
  Expected: "System.Globalization.HijriCalendar"
  But was:  "System.Globalization.GregorianCalendar"
  --------------------------------^

		   at DontLink.Calendars.CalendarTest.Hijri() in /Users/poupou/git/main/xamarin-macios/tests/linker/ios/dont link/CalendarTest.cs:line 28
	[FAIL] ThaiBuddhist :   Calendar
  Expected string length 41 but was 38. Strings differ at index 21.
  Expected: "System.Globalization.ThaiBuddhistCalendar"
  But was:  "System.Globalization.GregorianCalendar"
  --------------------------------^

		   at DontLink.Calendars.CalendarTest.ThaiBuddhist() in /Users/poupou/git/main/xamarin-macios/tests/linker/ios/dont link/CalendarTest.cs:line 35
	[FAIL] UmAlQura :   Calendar
  Expected string length 37 but was 38. Strings differ at index 21.
  Expected: "System.Globalization.UmAlQuraCalendar"
  But was:  "System.Globalization.GregorianCalendar"
  --------------------------------^

		   at DontLink.Calendars.CalendarTest.UmAlQura() in /Users/poupou/git/main/xamarin-macios/tests/linker/ios/dont link/CalendarTest.cs:line 21
DontLink.Calendars.CalendarTest : 226.39 ms
DontLink.Calendars : 229.61 ms

Configuration

dotnet SDK 6.0.100-preview.3.21202.5
the ICU enabled code is not yet merged - in part since I can't re-enable this test
unit test Dont link (where the linker is not used) shows the issue

Regression?

Yes. This is working on the current Xamarin.iOS SDK.

Other information

xamarin-macios tracking issue for globalization

Author:	spouliot
Assignees:	-
Labels:	`area-System.Globalization`, `untriaged`
Milestone:	-

spouliot · 2021-04-07T18:17:28Z

c.c. @steveisok

and re-enable some tests for dotnet Part of the fix for xamarin#8906 Known Issues * [some Calendar are not the expected ones](dotnet/runtime#50859) * [No support for tvOS](dotnet/runtime#50912)

and re-enable some tests for dotnet Part of the fix for #8906 Known Issues * [some Calendar are not the expected ones](dotnet/runtime#50859) * [No support for tvOS (bitcode)](dotnet/runtime#48508)

filipnavara · 2021-04-12T15:00:57Z

The tests themselves looks somewhat fishy to be honest. The actual expected mapping:

ar (~~Argentina, not~~ Arabia) -> System.Globalization.GregorianCalendar
ps (~~Persia~~ Pashto) -> System.Globalization.PersianCalendar
th (Thailand) -> System.Globalization.ThaiBuddhistCalendar

It doesn't explain why it would always return Gregorian calendar though.

spouliot · 2021-04-12T15:33:15Z

@filipnavara They are old tests we used to ensure DontLink and Link* tests gave the same results.

Being old does does not make them good (or bad) but I disagree with your interpretation of ar as "RFC 5646 Language Tags" shows

 'ar': 'Arabic',
  'ar-AE': 'Arabic (U.A.E.)',
  'ar-BH': 'Arabic (Bahrain)',
  'ar-DZ': 'Arabic (Algeria)',
  'ar-EG': 'Arabic (Egypt)',
  'ar-IQ': 'Arabic (Iraq)',
  'ar-JO': 'Arabic (Jordan)',
  'ar-KW': 'Arabic (Kuwait)',
  'ar-LB': 'Arabic (Lebanon)',
  'ar-LY': 'Arabic (Libya)',
  'ar-MA': 'Arabic (Morocco)',
  'ar-OM': 'Arabic (Oman)',
  'ar-QA': 'Arabic (Qatar)',
  'ar-SA': 'Arabic (Saudi Arabia)',
  'ar-SY': 'Arabic (Syria)',
  'ar-TN': 'Arabic (Tunisia)',
  'ar-YE': 'Arabic (Yemen)',

filipnavara · 2021-04-12T15:45:20Z

I stand corrected on the Arabic. Somehow I was totally convinced because "ar" is used as country code for Argentina (ISO 3166). Nevertheless the ar culture and most of its subcultures still return Gregorian calendar. ar-SA returns System.Globalization.UmAlQuraCalendar.

spouliot · 2021-04-12T15:50:43Z

yeah, like I said the tests were to ensure the same calendar was returned - whether or not the linker was enabled
now was mono correct to return that specific calendar (or not) is another story, one that might have evolved over time (at least for the .net desktop framework side)

filipnavara · 2021-04-12T16:16:14Z

So, apparently the ICU data filters are currently applied on all platforms for the app-local ICU builds and these filters (https://github.com/dotnet/icu/blob/313017d0effca6e8ade5fefc8e580224e65d8525/icu-filters/icudt.json#L261-L266) exclude non-Gregorian calendars from the data.

steveisok · 2021-04-12T17:21:52Z

Good catch. We built the filters initially w/ wasm in mind, so it looks like we'll have to tweak a little bit.

lewing · 2021-06-30T14:50:10Z

How important is ICU data size for iOS?

steveisok · 2021-06-30T14:56:51Z

Well, I haven't heard too much negative feedback re: the current ICU size ;-)

marek-safar · 2021-06-30T15:18:15Z

How important is ICU data size for iOS?

It's big enough already ;-)

tarekgh · 2021-06-30T16:35:01Z

The framework already has the code for the calendars. This looks to me the issue is just associating the calendar Id to the culture. which I guess can be supported with minimal increase of size.

lewing · 2021-07-01T03:11:53Z

We can either add the calendars back in or change the test, it is a size trade off. This appears to be marked p0 on iOS so I guess we need information from @spouliot

steveisok · 2021-07-13T04:55:35Z

Found we'll need to include "likelySubtags" in our filter in order for this to be fixed. Good news is that it only adds .1 MB to the dat file.

#124) This change also creates separate icudt filters for mobile and wasm as keeping likelySubtags results in a .1MB size increase. Addresses dotnet/runtime#50859

Adjust HttpClientHandler tests to cope with dotnet/runtime#50859 and #11392.

rolfbjarne · 2021-07-20T13:26:42Z

I'm seeing different results now with the latest .NET bump: xamarin/xamarin-macios@7e37fe6

filipnavara · 2021-07-20T13:33:06Z

You are seeing the correct results now, ie. same as net6.0 on desktop with system ICU.

steveisok · 2021-07-22T07:37:40Z

Assuming from the comments that this is fixed. Please reopen if you find otherwise.

dotnet-issue-labeler bot added area-System.Globalization untriaged New issue has not been triaged by the area owner labels Apr 7, 2021

steveisok added this to the 6.0.0 milestone Apr 7, 2021

steveisok self-assigned this Apr 7, 2021

tarekgh added os-ios Apple iOS and removed untriaged New issue has not been triaged by the area owner labels Apr 7, 2021

spouliot mentioned this issue Apr 8, 2021

[.NET 6] External dotnet issues xamarin/xamarin-macios#8901

Closed

61 tasks

spouliot mentioned this issue Apr 8, 2021

[dotnet] Add ICU support for iOS builds xamarin/xamarin-macios#11163

Merged

steveisok mentioned this issue Jul 13, 2021

Include likelySubtags and more calendar data for mobile configurations dotnet/icu#124

Merged

rolfbjarne added a commit to xamarin/xamarin-macios that referenced this issue Jul 20, 2021

[monotouch-test] Adjust HttpClientHandler tests.

7e37fe6

Adjust HttpClientHandler tests to cope with dotnet/runtime#50859 and #11392.

rolfbjarne added a commit to xamarin/xamarin-macios that referenced this issue Jul 20, 2021

[monotouch-test] Adjust HttpClientHandler tests.

5bd6106

Adjust HttpClientHandler tests to cope with dotnet/runtime#50859 and #11392.

steveisok closed this as completed Jul 22, 2021

ghost locked as resolved and limited conversation to collaborators Aug 21, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[iOS] Calendar not behaving correctly wrt culture #50859

[iOS] Calendar not behaving correctly wrt culture #50859

spouliot commented Apr 7, 2021

ghost commented Apr 7, 2021

Description

Configuration

Regression?

Other information

spouliot commented Apr 7, 2021

filipnavara commented Apr 12, 2021 •

edited

Loading

spouliot commented Apr 12, 2021

filipnavara commented Apr 12, 2021

spouliot commented Apr 12, 2021

filipnavara commented Apr 12, 2021 •

edited

Loading

steveisok commented Apr 12, 2021

lewing commented Jun 30, 2021

steveisok commented Jun 30, 2021

marek-safar commented Jun 30, 2021

tarekgh commented Jun 30, 2021

lewing commented Jul 1, 2021

steveisok commented Jul 13, 2021

rolfbjarne commented Jul 20, 2021

filipnavara commented Jul 20, 2021

steveisok commented Jul 22, 2021

[iOS] Calendar not behaving correctly wrt culture #50859

[iOS] Calendar not behaving correctly wrt culture #50859

Comments

spouliot commented Apr 7, 2021

Description

Configuration

Regression?

Other information

ghost commented Apr 7, 2021

Description

Configuration

Regression?

Other information

spouliot commented Apr 7, 2021

filipnavara commented Apr 12, 2021 • edited Loading

spouliot commented Apr 12, 2021

filipnavara commented Apr 12, 2021

spouliot commented Apr 12, 2021

filipnavara commented Apr 12, 2021 • edited Loading

steveisok commented Apr 12, 2021

lewing commented Jun 30, 2021

steveisok commented Jun 30, 2021

marek-safar commented Jun 30, 2021

tarekgh commented Jun 30, 2021

lewing commented Jul 1, 2021

steveisok commented Jul 13, 2021

rolfbjarne commented Jul 20, 2021

filipnavara commented Jul 20, 2021

steveisok commented Jul 22, 2021

filipnavara commented Apr 12, 2021 •

edited

Loading

filipnavara commented Apr 12, 2021 •

edited

Loading