panic: (6578): Can't change locale for LC_CTYPE(2)

[orgcontrib]

After a fresh install (via winget) on Windows 10 Pro with EU language configuration,..

C:\Users\devtest>ver

Microsoft Windows [Version 10.0.19045.3393]

C:\Users\devtest>perl -V
..\locale.c: 690: panic: (6578): Can't change locale for LC_CTYPE(2) from 'C' to 'English_Europe.65001'; errno=0

C:\Users\devtest>set LC_CTYPE
Environment variable LC_CTYPE not defined

C:\Users\devtest>winget show --id StrawberryPerl.StrawberryPerl
Found Strawberry Perl [StrawberryPerl.StrawberryPerl]
Version: 5.38.1
Publisher: strawberryperl.com project
Publisher Url: https://strawberryperl.com
Publisher Support Url: https://github.com/StrawberryPerl/Perl-Dist-Strawberry/issues
Author: strawberryperl.com project
Moniker: strawberryperl
Description:
  Perl is a programming language suitable for writing simple scripts as well as complex applications - see https://www.perl.org

  Strawberry Perl is a perl environment for MS Windows containing all you need to run and develop perl applications. It is designed to be as close as possible to perl environment on UNIX systems.

  It includes perl binaries, compiler (gcc) + related tools, all the external libraries (crypto, math, graphics, xml ...), all the bundled database clients and all you expect from Strawberry Perl.
Homepage: https://strawberryperl.com
License: Perl.org
License Url: https://github.com/StrawberryPerl/Perl-Dist-Strawberry/blob/master/LICENSE
Copyright Url: https://github.com/StrawberryPerl/Perl-Dist-Strawberry/blob/master/LICENSE
Release Notes Url: https://github.com/StrawberryPerl/Perl-Dist-Strawberry/releases/tag/SP_5380_5361
Tags:
  app
  development
  perl
  tools
Installer:
  Installer Type: wix
  Installer Locale: en-US
  Installer Url: https://github.com/StrawberryPerl/Perl-Dist-Strawberry/releases/download/SP_5380_5361/strawberry-perl-5.38.0.1-64bit.msi
  Installer SHA256: a9b44e50424dcc7e40b8f67d906c76a15469af3d5998e04635fa8465a0a56877
  Release Date: 2023-07-11

C:\Users\devtest>winget --info
Windows Package Manager v1.5.2201
Copyright (c) Microsoft Corporation. All rights reserved.

Windows: Windows.Desktop v10.0.19045.3393
System Architecture: X64
Package: Microsoft.DesktopAppInstaller v1.20.2201.0

Winget Directories
-----------------------------------------------------------------------------------------------------------------------
Logs                               %LOCALAPPDATA%\Packages\Microsoft.DesktopAppInstaller_8wekyb3d8bbwe\LocalState\Diag…
User Settings                      %LOCALAPPDATA%\Packages\Microsoft.DesktopAppInstaller_8wekyb3d8bbwe\LocalState\sett…
Portable Links Directory (User)    %LOCALAPPDATA%\Microsoft\WinGet\Links
Portable Links Directory (Machine) C:\Program Files\WinGet\Links
Portable Package Root (User)       %LOCALAPPDATA%\Microsoft\WinGet\Packages
Portable Package Root              C:\Program Files\WinGet\Packages
Portable Package Root (x86)        C:\Program Files (x86)\WinGet\Packages

Links
---------------------------------------------------------------------------
Privacy Statement   https://aka.ms/winget-privacy
License Agreement   https://aka.ms/winget-license
Third Party Notices https://aka.ms/winget-3rdPartyNotice
Homepage            https://aka.ms/winget
Windows Store Terms https://www.microsoft.com/en-us/storedocs/terms-of-sale

Admin Setting                             State
--------------------------------------------------
LocalManifestFiles                        Disabled
BypassCertificatePinningForMicrosoftStore Disabled
InstallerHashOverride                     Disabled
LocalArchiveMalwareScanOverride           Disabled

C:\Users\devtest>

[orgcontrib]

A manual (temporary) intervention doesn't help.

C:\Users\devtest>set LC_CTYPE="English_Europe.65001"

C:\Users\devtest>set LC_CTYPE
LC_CTYPE="English_Europe.65001"

C:\Users\devtest>perl -V
..\locale.c: 690: panic: (6578): Can't change locale for LC_CTYPE(2) from 'C' to 'English_Europe.65001'; errno=0

C:\Users\devtest>

[orgcontrib]

Easiest workaround is to switch to a well-known language config (i.e. English UK). Nevertheless, this should be fixed.

C:\Users\devtest>perl -V
Summary of my perl5 (revision 5 version 38 subversion 0) configuration:

  Platform:
    osname=MSWin32
    osvers=10.0.17763.4499
    archname=MSWin32-x64-multi-thread
    uname='Win32 strawberry-perl 5.38.0.1 # 12:34:49 Thu July 06 2023 x64'
    config_args='undef'
    hint=recommended
    useposix=true
    d_sigaction=undef
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=undef
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
  Compiler:
    cc='gcc'
    ccflags =' -DWIN32 -DWIN64 -DPERL_TEXTMODE_SCRIPTS -DMULTIPLICITY -DPERL_IMPLICIT_SYS -DUSE_PERLIO -D__USE_MINGW_ANSI_STDIO -fwrapv -fno-strict-aliasing -mms-bitfields'
    optimize='-Os'
    cppflags='-DWIN32'
    ccversion=''
    gccversion='13.1.0'
    gccosandvers=''
    intsize=4
    longsize=4
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='long long'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='g++'
    ldflags ='-s -L"C:\STRAWB~1\perl\lib\CORE" -L"C:\STRAWB~1\c\lib" -L"C:\STRAWB~1\c\x86_64-w64-mingw32\lib" -L"C:\STRAWB~1\c\lib\gcc\x86_64-w64-mingw32\13.1.0"'
    libpth=C:\STRAWB~1\c\lib C:\STRAWB~1\c\x86_64-w64-mingw32\lib C:\STRAWB~1\c\lib\gcc\x86_64-w64-mingw32\13.1.0 C:\STRAWB~1\c\x86_64-w64-mingw32\lib C:\STRAWB~1\c\lib\gcc\x86_64-w64-mingw32\13.1.0
    libs= -lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32 -lmpr -lwinmm -lversion -lodbc32 -lodbccp32 -lcomctl32
    perllibs= -lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32 -lmpr -lwinmm -lversion -lodbc32 -lodbccp32 -lcomctl32
    libc=
    so=dll
    useshrplib=true
    libperl=libperl538.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_win32.xs
    dlext=xs.dll
    d_dlsymun=undef
    ccdlflags=' '
    cccdlflags=' '
    lddlflags='-shared -s -L"C:\STRAWB~1\perl\lib\CORE" -L"C:\STRAWB~1\c\lib" -L"C:\STRAWB~1\c\x86_64-w64-mingw32\lib" -L"C:\STRAWB~1\c\lib\gcc\x86_64-w64-mingw32\13.1.0"'


Characteristics of this binary (from libperl):
  Compile-time options:
    HAS_LONG_DOUBLE
    HAS_TIMES
    HAVE_INTERP_INTERN
    MULTIPLICITY
    PERLIO_LAYERS
    PERL_COPY_ON_WRITE
    PERL_DONT_CREATE_GVSV
    PERL_HASH_FUNC_SIPHASH13
    PERL_HASH_USE_SBOX32
    PERL_IMPLICIT_SYS
    PERL_MALLOC_WRAP
    PERL_OP_PARENT
    PERL_PRESERVE_IVUV
    PERL_USE_SAFE_PUTENV
    USE_64_BIT_INT
    USE_ITHREADS
    USE_LARGE_FILES
    USE_LOCALE
    USE_LOCALE_COLLATE
    USE_LOCALE_CTYPE
    USE_LOCALE_NUMERIC
    USE_LOCALE_TIME
    USE_PERLIO
    USE_PERL_ATOF
  Built under MSWin32
  Compiled at Jul  6 2023 22:36:11
  @INC:
    C:/Strawberry/perl/site/lib
    C:/Strawberry/perl/vendor/lib
    C:/Strawberry/perl/lib

C:\Users\devtest>

[sisyphus]

I'm not well-informed when it comes to "locale" matters, but the following might help point people in the right direction regarding the 'English_Europe.65001' locale.
On StrawberryPerl-5.38.0:

D:\>perl -MPOSIX -wle "$loc = POSIX::setlocale( LC_ALL, 'English_Europe.65001' ); print $]; print $loc;"
5.038000
Use of uninitialized value $loc in print at -e line 1.

Apparently, Strawberry Perl has no awareness of the 'English_Europe.65001' locale setting.
I find the same with my own build of perl-5.38.0:

D:\>perl -MPOSIX -wle "$loc = POSIX::setlocale( LC_ALL, 'English_Europe.65001' ); print $]; print $loc;"
5.038000
Use of uninitialized value $loc in print at -e line 1.

It's to be expected that those 2 builds of perl would produce the same result. They were built from the same perl source, with essentially the same configuration options, using the same mingw-w64 (gcc-13.1.0) toolchain.

But things change for the better when I switch to my build of perl-5.39.2 (built by mingw-w64 port of gcc-13.2.0):

D:\>perl -MPOSIX -wle "$loc = POSIX::setlocale( LC_ALL, 'English_Europe.65001' ); print $]; print $loc;"
5.039002
English_Europe.utf8

(I assume that 'English_Europe.utf8' and 'English_Europe.65001' are the same thing ??)
I don't know if that "success" arises from corrections made in the perl-5.39.2 source, or from corrections made in gcc-13.2.0.

I do know that, with perl-5.38.0 built by Visual Studio 2022, I get:

D:\>perl -MPOSIX -wle "$loc = POSIX::setlocale( LC_ALL, 'English_Europe.65001' ); print $];print $loc;"
5.038000
English_Europe.utf8

Also, if I build perl-5.39.2 with gcc-13.1.0, I get the same result as produced by 5.38.0 built with gcc-13.1.0

D:\comp-1310\perl-5.39.2\win32>..\perl -I..lib -MPOSIX -wle "$loc = POSIX::setlocale( LC_ALL, 'English_Europe.65001' ); print $]; print $loc;"
5.039002
Use of uninitialized value $loc in print at -e line 1.

To me, this is a strong indication that the "success" with the 5.39.2 mingw-build is due to changes made to the mingw-w64 port of gcc-13.2.0, not to any changes made to the perl source.

IOW, I think we can say that it's "fixed in gcc-13.2.0" - though, FAIK, that fix might actually be in the runtime library, rather than gcc itself.

UPDATE: To elaborate a little on my referring to "runtime version", I should point out that my gcc-13.1.0 is "msvcrt", but the gcc-13.2.0 is "ucrt".

Cheers,
Rob

[shawnlaffan]

Thanks @orgcontrib for reporting and @sisyphus for the diagnosis.

I'll try with gcc-13.2 when next I run a build (hopefully before end September). If it is not fixed with msvcrt gcc-13.2 then it might need changes at the perl level.

[sisyphus]

Curiosity got the better of me, yet again.
I tried this simple C program:

/* locale.c */
#include <locale.h>
#include <stdio.h>

int main(int argc, char *argv[]) {
    const char *locale = setlocale(LC_ALL, argv[1]);
    printf("%s\n", locale);
    return 0;
}

With gcc version 13.2.0 (MinGW-W64 x86_64-msvcrt-posix-seh, built by Brecht Sanders)

D:\C>gcc -o locale.exe locale.c

D:\C>locale.exe German.utf8
(null)

D:\C>locale.exe English_Europe.65001
(null)

With gcc version 13.2.0 (MinGW-W64 x86_64-ucrt-mcf-seh, built by Brecht Sanders)

D:\C>gcc -o locale.exe locale.c

D:\C>locale.exe German.utf8
German_Germany.utf8

D:\C>locale.exe English_Europe.65001
English_Europe.utf8

Looks like you'll need to use a "ucrt" build to get the good behaviour.
Admittedly, the 2 compilers I've used there also differ in the threads model, but I think that's of no significance. (I couldn't find a MinGW-W64 x86_64-msvcrt-mcf-seh build of gcc-13.2.0.)

Perl-5.38.0 source won't build straight out of the box with a "ucrt" compiler, and I'm not sure what patches would be needed.
I believe that most of the required changes had been built into the 5.38.0 source, so the necessary patching might not be extensive,
Perl-5.39.2 and later do build straight out of the box with a "ucrt" compiler.
If we stay alert we can probably get that fix back-ported to 5.38.1.

Cheers,
Rob

[FrancescElies]

After installing perl from the website (Latest Release: 5.38.2.2 (2023-12-11))

PS C:\Users\FrancescElies> perl --version
..\locale.c: 690: panic: (6578): Can't change locale for LC_CTYPE(2) from 'C' to 'English_Europe.65001'; errno=0

Changed Language to english uk but still having the same issue so far.

Suggestions?
For the time being I will try with older releases.

[shawnlaffan]

This will be fixed when 5.40 is built and released as it will use UCRT instead of MSCVRT.

It's a matter of finding the time to run the build.

If you want to test with a perl RC build then one can be accessed from https://github.com/StrawberryPerl/Perl-Dist-Strawberry/releases/tag/dev_5.40.0_RC1_UCRT

[bundgaard]

I have it using the msvc, so will these changes support msvc, as I get strawberry using vcpkg?

[shawnlaffan]

SP 5.40.0.1 has now been released. This uses UCRT so this problem should be resolved now.

The release can be downloaded from https://github.com/StrawberryPerl/Perl-Dist-Strawberry/releases/tag/SP_54001_64bit_UCRT

A 5.38.x UCRT build might be released but it can be done under a separate issue.

I'll close this in a few days, or when next I do a sweep of issues.

[shawnlaffan]

Closing as noted.