Make polynomial rings collectable after arithmetic operations

[simon-king-jena]

By #13447, polynomial rings can be garbage collected. However, they cannot if its elements were involved in an arithmetic operation. So, apparently some strong reference chain is created in the coercion system.

The purpose of this ticket is to find that reference chain and invent means to remove it. Perhaps related with #26811?

Depends on #13447

CC: @jdemeyer

Component: memleak

Issue created by migration from https://trac.sagemath.org/ticket/27095

[simon-king-jena]

comment:1

Here is verification that in principle polynomial rings can be garbage collected:

sage: n = 0
sage: import gc
sage: while 1:
....:     if n%100 == 0: print(get_memory_usage())
....:     R = QQ['a{}'.format(n), 'b{}'.format(n)]
....:     p = R.1
....:     del R, p
....:     _ = gc.collect()
....:     n += 1
....:     
7332.671875
7332.671875
7332.671875
7332.671875
7332.671875
7332.671875
...

And here is evidence that they cannot if an arithmetic operation occurs. I am studying different operations, each time in a new session, just to be on the safe side:

1. Scalar multiplication:

   sage: n = 0
   sage: import gc
   sage: while 1:
   ....:     if n%100 == 0: print(get_memory_usage())
   ....:     R = QQ['a{}'.format(n), 'b{}'.format(n)]
   ....:     p = 2*R.1
   ....:     del R, p
   ....:     _ = gc.collect()
   ....:     n += 1
   ....:     
   7332.6640625
   7333.6640625
   7333.9140625
   7334.04296875
   7334.04296875
   7334.04296875
   7334.04296875
   ...
   

   --> Interestingly, after an increase, the memory consumption remains steady.

2. Addition

   sage: n = 0
   sage: import gc
   sage: while 1:
   ....:     if n%100 == 0: print(get_memory_usage())
   ....:     R = QQ['a{}'.format(n), 'b{}'.format(n)]
   ....:     p = R.1+R.0
   ....:     del R, p
   ....:     _ = gc.collect()
   ....:     n += 1
   ....:     
   7340.65625
   7340.65625
   7340.65625
   7340.65625
   7340.65625
   7340.65625
   7340.65625
   7340.65625
   ...
   

   --> No leak here.

3. Multiplication

   sage: n = 0
   sage: import gc
   sage: while 1:
   ....:     if n%100 == 0: print(get_memory_usage())
   ....:     R = QQ['a{}'.format(n), 'b{}'.format(n)]
   ....:     p = R.1*R.0
   ....:     del R, p
   ....:     _ = gc.collect()
   ....:     n += 1
   ....:     
   7332.6328125
   7332.6328125
   7332.6328125
   7332.6328125
   7332.6328125
   7332.6328125
   7332.6328125
   ...
   

   --> No leak.

So, apparently the leak occurs in the action of the base ring on the polynomial ring.

[nbruin]

comment:2

See #27083 . Increasing memory usage that flattens off is not an indication of a memory leak. It may even be the case (if CachedRepresentation gets used) that objects remain in memory without apparent reason due to its recently acquired strong reference cache.

You would want to check that no such cache is in effect here. If it is, you'd need to check that there are more than 128 polynomial rings remaining in memory at some point. And I think you'd want to see if they are on the python heap using the usual gc.get_objects. If they're not it would likely be a memleak in libsingular instead.

Of course the leak you describe (if it exists!) is almost certainly on the python side, but finding the offending objects on the heap and backtracing their references should be a pretty quick way of finding where the leak occurs. The graphs produced by objgraph.backref have helped me significantly in the process. Nowadays: ignore references from a tuple with 128 elements. That's due to #24954 .

[embray]

comment:3

Ticket retargeted after milestone closed (if you don't believe this ticket is appropriate for the Sage 8.8 release please retarget manually)

[embray]

comment:4

As the Sage-8.8 release milestone is pending, we should delete the sage-8.8 milestone for tickets that are not actively being worked on or that still require significant work to move forward. If you feel that this ticket should be included in the next Sage release at the soonest please set its milestone to the next release milestone (sage-8.9).