BUG: groupby().agg( ) with min/max on Int64 leads to incorrect results

[ppletscher]

• [x ] I have checked that this issue has not already been reported.

• [x ] I have confirmed this bug exists on the latest version of pandas.

• (optional) I have confirmed this bug exists on the master branch of pandas.

---

Code Sample, a copy-pastable example

import pandas as pd

ts = 1618556707013635762
pdf = pd.DataFrame([{"id": 2, "ts": ts}, {"id": 2, "ts": ts+1}])
pdf["ts"] = pd.array(pdf["ts"], dtype="Int64")
pdf.groupby("id").agg({"ts": "min"})

Problem description

The above code will output 1618556707013635840 as the minimum which is incorrect.

Expected Output

Minimum should be 1618556707013635762. I suspect there is a cast happening somewhere during the aggregation. If we keep the ts column at int64 (i.e. use the non-nullable type) the result is as expected.

Output of pd.show_versions()

INSTALLED VERSIONS

commit : 2cb9652
python : 3.9.5.final.0
python-bits : 64
OS : Linux
OS-release : 5.11.7-200.fc33.x86_64
Version : #1 SMP Wed Mar 17 18:55:20 UTC 2021
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL :
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 1.2.4
numpy : 1.20.3
pytz : 2021.1
dateutil : 2.8.1
pip : 21.1.1
setuptools : 56.0.0
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : 7.24.0
pandas_datareader: None
bs4 : None
bottleneck : None
fsspec : None
fastparquet : None
gcsfs : None
matplotlib : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
numba : None

Potentially related issues

The issues below seem related, but from what I can see are slightly different:

• ENH: improve the resulting dtype for groupby operations on nullable dtypes #37494 seems like a more general problem where it is a priori unclear what the resulting dtype of the output column should be. For the min and max I would expect to get the same dtype as in the input column.
• BUG: groupby mean/median/var should always be float values #41137 For the present issue we actually want to keep the original precision and not cast to float.

[jreback]

try this in master as pretty sure this is fixed

[ppletscher]

Thanks a lot for the quick reply.

I've just tried this inside a Docker build as described here and unfortunately I also see the same behaviour there.

pd.show_versions() shows the following:

INSTALLED VERSIONS
------------------
commit           : bc9b470711cd2126d0a834186103a4ea2215f7eb
python           : 3.8.10.final.0
python-bits      : 64
OS               : Linux
OS-release       : 5.11.7-200.fc33.x86_64
Version          : #1 SMP Wed Mar 17 18:55:20 UTC 2021
machine          : x86_64
processor        : x86_64
byteorder        : little
LC_ALL           : C.UTF-8
LANG             : C.UTF-8
LOCALE           : en_US.UTF-8

pandas           : 1.3.0.dev0+1766.gbc9b470711
numpy            : 1.20.3
pytz             : 2021.1
dateutil         : 2.8.1
pip              : 21.1.2
setuptools       : 49.6.0.post20210108
Cython           : 0.29.23
pytest           : 6.2.4
hypothesis       : 6.13.10
sphinx           : 3.5.4
blosc            : None
feather          : None
xlsxwriter       : 1.4.3
lxml.etree       : 4.6.3
html5lib         : 1.1
pymysql          : None
psycopg2         : None
jinja2           : 2.11.3
IPython          : 7.24.0
pandas_datareader: None
bs4              : 4.9.3
bottleneck       : 1.3.2
fsspec           : 2021.05.0
fastparquet      : 0.6.3
gcsfs            : 2021.05.0
matplotlib       : 3.4.2
numexpr          : 2.7.3
odfpy            : None
openpyxl         : 3.0.7
pandas_gbq       : None
pyarrow          : 4.0.0
pyxlsb           : None
s3fs             : 0.4.2
scipy            : 1.6.3
sqlalchemy       : 1.4.17
tables           : 3.6.1
tabulate         : 0.8.9
xarray           : 0.18.2
xlrd             : 2.0.1
xlwt             : 1.3.0
numba            : 0.53.1

[mzeitlin11]

Thanks for trying this in master! This cast was avoided for cummin/max in #40651, still needs to be done for min/max, xref #37493