Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unhandled exception: NoneType #820

Closed
jim-moe opened this issue Oct 9, 2017 · 49 comments · Fixed by #1599
Closed

Unhandled exception: NoneType #820

jim-moe opened this issue Oct 9, 2017 · 49 comments · Fixed by #1599
Assignees
Labels
Bug Heisenbug a problem that is not reproducible but random (non-deterministic)
Milestone

Comments

@jim-moe
Copy link

jim-moe commented Oct 9, 2017

opensuse 42.2
linux 4.4.87-18.29-default x86_64
backintime 1.1.20

I occasionally receive the error message below during a backup. Probably not a Good Thing.

Unhandled exception in thread started by <function __log_keyring_warning at 0x7fd39670a400>
Traceback (most recent call last):
File "/usr/share/backintime/common/tools.py", line 1463, in __log_keyring_warning
TypeError: 'NoneType' object is not callable

@buhtz
Copy link
Member

buhtz commented Oct 12, 2017

Can you reproduce this error while running bit with --debug option and post the complete output?

@jim-moe
Copy link
Author

jim-moe commented Oct 12, 2017

The fault happens very rarely. How much output does --debug produce?

@buhtz
Copy link
Member

buhtz commented Oct 14, 2017

Not very much. The output appears on the console. So you have to start bit on shell.

@Germar Would the --debug output appear in any logfile if that option would be added to the bit-call in the crontab?

Did you copy and paste the error message? I can not find __log_keyring_warning anywhere in the code.

@Germar
Copy link
Member

Germar commented Oct 14, 2017

Sure. Debugging messages from --debug will appear in the default log.

__log_keyring_warning is in line 1457 in version 1.1.20. But the line 1463 which is reported in the Traceback is not part of that function. Did you modify tools.py by your self? Can you please check with a vanilla BiT install?

@jim-moe
Copy link
Author

jim-moe commented Oct 16, 2017

No, I have made no changes whatsoever to the source. BiT was installed from the opensuse repository, v1.1.20-3.3.1. I presume you submit the application for inclusion.

-rw-r--r-- 1 root root 50750 Apr 21 00:25 tools.py
-rw-r--r-- 1 root root 11202 Sep 9 2012 tools.pyc

@jim-moe
Copy link
Author

jim-moe commented Oct 27, 2017

The error finally recurred. There was nothing in the BiT log file itself that contained "keyring."

The command line:
/usr/bin/nice -n19 /usr/bin/ionice -c2 -n7 /usr/bin/backintime --debug --profile-id 2 backup-job >/dev/null

The additional output:
Unhandled exception in thread started by <function __log_keyring_warning at 0x7fd698dbf400>
Traceback (most recent call last):
File "/usr/share/backintime/common/tools.py", line 1463, in __log_keyring_warning
File "", line 2237, in _find_and_load
File "", line 2222, in _find_and_load_unlocked
File "", line 2150, in _find_spec
TypeError: 'NoneType' object is not callable

@pclinuxer
Copy link

Hello:
Yes, I did mention that I had found #820 when I posted #1169 in 07/2021.
#1169 being closed for being a duplicate of #820, makes total sense.

But I see that #820 has also been closed but there's no mention of a fix for it.
I cannot find a reference to it in the link to Codeberg-AsGithubAlternative-buhtz.

Please advise.

Thanks in advance.
Best,

PCL

@buhtz
Copy link
Member

buhtz commented Aug 30, 2022

I am confused also. ;)

We are here at Issue 820 which is open. But you say that #820 was closed?

@pclinuxer
Copy link

Hello:

Hmm ...
Yes, I did.
My mistake.

Too early, not enough coffee. 8^°
I saw the Closed with the purple background and ...

Sorry.
Thanks for the heads up.

Best,

PCL

@buhtz
Copy link
Member

buhtz commented Sep 20, 2022

Related Debian report
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=990343

@buhtz
Copy link
Member

buhtz commented Sep 25, 2022

@jim-moe There is a new forming maintaining team and we do review all issues. Is this problem still relevant for you or did you find a solution?

@pclinuxer
Copy link

pclinuxer commented Sep 25, 2022

Hello:
@buhtzz FWIW, I am still getting the warning from Cron in my mbox.
Like I mentioned, it happens every so often but have not been able to figure out under what conditions so cannot reproduce it.

Unhandled exception in thread started by <function __log_keyring_warning at 0x7ff1f8266268>
Traceback (most recent call last):
  File "/usr/share/backintime/common/tools.py", line 1458, in __log_keyring_warning
TypeError: 'NoneType' object is not callable

The news is that I am now getting this warning, which I did not get before:

WARNING: import keyring failed
WARNING: import keyring failed
WARNING: import keyring failed

In any case, BiT seems to be doing its job and I don't know if it is a keyring issue or not.
Wanted to let you know in case it is relevant.

Best,

JHM

@buhtz
Copy link
Member

buhtz commented Sep 25, 2022

Maybe it helps when you install one of the python-keyring packages?

@pclinuxer
Copy link

pclinuxer commented Sep 25, 2022

Hello:
Thanks for the fast reply.

Here is what I have installed on my Devuan installation:

user@devuan:~$ apt list | grep installed | grep keyring
--- snip ---
debian-archive-keyring/oldstable,oldstable,now 2019.1+deb10u1 all [installed]
devuan-keyring/oldstable,oldstable,now 2022.09.04 all [installed]
gir1.2-gnomekeyring-1.0/now 3.12.0-1+b2 amd64 [installed,local]
gnome-keyring-pkcs11/oldstable,now 3.28.2-5 amd64 [installed,automatic]
gnome-keyring/oldstable,now 3.28.2-5 amd64 [installed,automatic]
libgnome-keyring-common/now 3.12.0-1 all [installed,local]
libgnome-keyring0/now 3.12.0-1+b2 amd64 [installed,local]
libpam-gnome-keyring/oldstable,now 3.28.2-5 amd64 [installed,automatic]  

python-keyring/oldstable,oldstable,now 17.1.1-1 all [installed,automatic]
python-keyrings.alt/oldstable,oldstable,now 3.1.1-1 all [installed,automatic]
python3-keyring/oldstable,oldstable,now 17.1.1-1 all [installed,automatic]
python3-keyrings.alt/oldstable,oldstable,now 3.1.1-1 all [installed,automatic]

user@devuan:~$ 

At least, it is what I have available in the Devuan repository, the installation being up to date:

user@devuan:~$ apt list | grep python | grep keyring
--- snip ---
python-keyring/oldstable,oldstable,now 17.1.1-1 all [installed,automatic]
python-keyrings.alt/oldstable,oldstable,now 3.1.1-1 all [installed,automatic]
python3-keyring/oldstable,oldstable,now 17.1.1-1 all [installed,automatic]
python3-keyrings.alt/oldstable,oldstable,now 3.1.1-1 all [installed,automatic]
user@devuan:~$ 

After having some espresso, I recalled that I originally attempted to find an answer on the Devuan forum last year and what I got answers led me to post the bug report here.

If you do a search with the text WARNING: import keyring failed you get (ggl) 61 results.
It seems that BiT is involved in quite a few, some as far back as 2015.

Thanks for your input.

Best,

JHM

@pclinuxer
Copy link

Hello:

Here is what I have installed on my Devuan installation ...

I have checked and have not been able to find a good reason for having gnome-keyring in my Devuan installation.

[root@devuan ~]# aptitude why gnome-keyring
i   pcmanfm       Recommends gvfs-backends
i A gvfs-backends Recommends gnome-keyring
[root@devuan ~]# 

Or for gvfs-backends for that matter.

They are not dependencies, they are pcmanfm recommends.
And we all know that recommends can sometimes be silently problematic.

Just shooting in the dark, but ...
I wonder if the problem isn't being caused by having too many ***-keyring instances?

One way to find out:
Today at noon (03:00 GMT) my rig was purged of gnome-keyring and gvfs-backends.
Now let's wait and see if I get any more mail in my mbox regarding BiT or whatever keyring.

I'll post back if I get another message or in a fortnight, whatever comes first.

Best,

JHM

@jim-moe
Copy link
Author

jim-moe commented Sep 25, 2022

@jim-moe There is a new forming maintaining team and we do review all issues. Is this problem still relevant for you or did you find a solution?
I am so glad to hear that BiT is being maintained again!

I have not seen this error, NoneType, in a very long time (years).

@pclinuxer
Copy link

pclinuxer commented Sep 26, 2022

Hello:

I'll post back if I get another message or in a fortnight, whatever comes first.

BiT apparently is working properly and cron has not sent me any mails.
But to check, I ran it manually and this is what I got:

user@devuan:~$ backintime backup
Gkr-Message: 08:42:53.971: secret service operation failed: The name org.freedesktop.secrets was not provided by any .service files

Back In Time
Version: 1.1.24

Back In Time comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to redistribute it
under certain conditions; type `backintime --license' for details.

INFO: Lock
WARNING: Inhibit Suspend failed.
WARNING: import keyring failed
INFO: Take a new snapshot. Profile: 1 Main profile
INFO: [qt4systrayicon] begin loop
INFO: Call rsync to take the snapshot
INFO: Save config file
INFO: Create info file
INFO: Remove backups older than: 20220117-000000
INFO: Keep min free disk space: 40960 MiB
INFO: Keep min 5% free inodes
INFO: Unlock
user@devuan:~$ INFO: [qt4systrayicon] end loop

It would seem that BiT does not need gnome-keyring to work properly, but from the manual backup terminal's output we can see that the import keyring failed warning is still there.
ie: whether gnome-keyring is installed or not, so it does not seem to be the what causes the BiT problem.

Any ideas?

Edit:
Reinstalling gnome-keyring got rid of the import keyring failed warning and the manual backup went on as expected.
ie: import keyring did not fail.

But a cron job to run BiT was logging the error message related to keyring described previously.

--> Q: What links BiT and gnome-keyring and why does a cron job running BiT want to import it?

I searched yet some more and found a lead of sorts here:

The OP had purged gnome-keyring and was getting errors in his logs.

... realized that the problem was not limited just to Docker, any services using libsecret were broken. As I use KeePassXC with libsecret integration I took a look in the settings and it turns out the integration was disabled. I purged the gnome-keyring package again, killed the gnome-keyring service and enabled the KeePassXC integration and it all worked as usual.

Thought I'd post all this as it may have some bearing ie: the libsecret integration.

Thanks in advance.

Best,

PCL

@aryoda
Copy link
Contributor

aryoda commented Oct 7, 2022

Maybe related to #1321 (fixing there)...

--> Q: What links BiT and gnome-keyring and why does a cron job running BiT want to import it?

You can save your password for your private SSH key and EncFS password (local encryption) in a supported keyring in the BiT GUI:

Settings GUI > General > Mode: SSH > checkbox "Save Password to Keyring"

If the job is scheduled in the settings it is added to cron and it may request this password via the keyring python package that tries to access your installed "password safe".

Note: It is important to configure a supported keyring backend in a config file if ChainedBackend (keyring.backends.chainer.ChainerBackend) is shown in the debug output of BiT. See the linked issue for an example and where to find the config file...

@pclinuxer
Copy link

Hello:

I have not had this issue again since my Sept 25th. post and the usual was once every week or so.
Totally random.

FWIW, I had purged and the reinstalled gnome-keyring before posting.
Maybe there's a clue there?

Like I mentioned before, BiT was doing its job properly (as far as I can tell) in spite of the error.
And I have my settings as > General > Mode: Local as my BiT backups go to a local drive. (yes, I know ... )
It is a repeated schedule which runs once a day because the box run regularly but not always, which then makes it sort of irregular. 8^°

Thanks for your input.

Best,

PCL

@emtiu emtiu added the Feedback needs user response, may be closed after timeout without a response label Oct 7, 2022
@emtiu
Copy link
Member

emtiu commented Oct 7, 2022

To everyone involved: Would you say this Issue can be closed?

@pclinuxer
Copy link

Hello:

To everyone involved: Would you say this Issue can be closed?
I may be the last one who actually saw this problem in the past couple of years.
Problem without apparent consequences but still a problem.
ie: when there's an error and it gets logged, it is a problem.

I would suggest leaving it open till the end of this month so I see if it crops up again.
If it does not, I think we can call it 'case closed'.

Unless it crops up again. 8^°

I sort of don't like 1. not knowing exactly what caused it, 2. think that it got solved the MS way.
ie: by reinstalling something.

Thanks,
PCL

@emtiu
Copy link
Member

emtiu commented Oct 7, 2022

Agreed, we'll give this Issue a 4-week timeout.

@pclinuxer
Copy link

Hello:

Right, thank you.
I've made a note to be back 04/10.
That or report before that date if anything crops up.

Best,

PCL

@pclinuxer
Copy link

Hello:

... or report before that date if anything crops up.
I'm sorry to have to report that I have had another notification in my mbox:

From groucho@devuan Wed Oct 12 23:00:01 2022
Received: from groucho (uid 1000)
	(envelope-from groucho@devuan)
	id df3b4
	by devuan (DragonFly Mail Agent v0.11);
	Wed, 12 Oct 2022 23:00:01 -0300
From: root (Cron Daemon)
To: groucho
Subject: Cron <groucho@devuan> /usr/bin/nice -n 19 /usr/bin/ionice -c2 -n7 /usr/bin/backintime backup-job >/dev/null
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 8bit
X-Cron-Env: <SHELL=/bin/sh>
X-Cron-Env: <PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin>
X-Cron-Env: <HOME=/home/groucho>
X-Cron-Env: <LOGNAME=groucho>
Date: Wed, 12 Oct 2022 23:00:01 -0300
Message-Id: <63477121.df3b4.3077b0cf@devuan>

Unhandled exception in thread started by <function __log_keyring_warning at 0x7f137fbe2488>
Traceback (most recent call last):
File "/usr/share/backintime/common/tools.py", line 1458, in __log_keyring_warning
TypeError: 'NoneType' object is not callable

As you can see, the error is the same.
/usr/share/backintime/common/tools.py -> Line 1458

These are the last 10 lines of that particular script:

--- snip ---
1457 def __log_keyring_warning():
1458     from time import sleep
1459     sleep(0.1)
1460     logger.warning('import keyring failed')
1461 
1462 if keyring is None and keyring_warn:
1463     #delay warning to give logger some time to import
1464     import _thread
1465     _thread.start_new_thread(__log_keyring_warning, ())
1466     # logger.warning('import keyring failed')
1467 

So I'd say this issue should remain open.

My box run Linux Devuan Beowulf with a backported kernel:

groucho@devuan:~$ uname -a
Linux devuan 5.10.0-0.deb10.16-amd64 #1 SMP Debian 5.10.127-2~bpo10+1 (2022-07-28) x86_64 GNU/Linux
groucho@devuan:~$ 

BiT 1.1.24.0

groucho@devuan:~$ apt list | grep installed | grep backin
--- snip ---
backintime-common/oldstable,oldstable,now 1.1.24-0.1 all [installed]
backintime-qt4/oldstable,oldstable,now 1.1.24-0.1 all [installed]
groucho@devuan:~$ 

Best,

PCL

@aryoda
Copy link
Contributor

aryoda commented Oct 13, 2022

File "/usr/share/backintime/common/tools.py", line 1458, in __log_keyring_warning
TypeError: 'NoneType' object is not callable

1457 def __log_keyring_warning():
1458     from time import sleep
1459     sleep(0.1)
1460     logger.warning('import keyring failed')
1461 
1462 if keyring is None and keyring_warn:
1463     #delay warning to give logger some time to import
1464     import _thread
1465     _thread.start_new_thread(__log_keyring_warning, ())
1466     # logger.warning('import keyring failed')
1467 

May be a multi-threading issue

Could you move the line 1458 after 1464 so that the sleep function is imported before the thread starts?...

Then wait and see ;-)

Edit: If it fails you could try a variant: import time + time.sleep(0.1) in the original code lines and if this does not work as another variation move the import again to after 1464

@pclinuxer
Copy link

pclinuxer commented Oct 13, 2022

Hello:

Thanks for the prompt reply. 8^)

May be a multi-threading issue
I see ...

Could you move the line 1458 after 1464 so that the sleep function is imported before the thread starts?...
Right, I'll do that and see if the issue crops up again.
Like I mentioned, it is totally random and sometimes does no happen for the longest while.

Then wait and see ;-)
Edit: If it fails you could try ...

Better if I report back when/if I get another notification and get further instructions.
I don't want to muck-up my BiT config. 8^°
We'll have to give it a min of 15 days and a max of 30.
If in four weeks there is no notification, we could consider that it may have worked.

Just to check, is this it?:

1457 def __log_keyring_warning():
1459     sleep(0.1)
1460     logger.warning('import keyring failed')
1461 
1462 if keyring is None and keyring_warn:
1463     #delay warning to give logger some time to import
1464     import _thread
1458     from time import sleep
1465     _thread.start_new_thread(__log_keyring_warning, ())
1466     # logger.warning('import keyring failed')
1467 

Obviously no renumbering needed as these are generated by jed.

Thanks for your input.

Best,

PCL

@aryoda
Copy link
Contributor

aryoda commented Oct 13, 2022

Just to check, is this it?:

1457 def __log_keyring_warning():
1459     sleep(0.1)
1460     logger.warning('import keyring failed')
1461 
1462 if keyring is None and keyring_warn:
1463     #delay warning to give logger some time to import
1464     import _thread
1458     from time import sleep
1465     _thread.start_new_thread(__log_keyring_warning, ())
1466     # logger.warning('import keyring failed')
1467 

Exactly like I described it! I have just checked it (using Git master branch - but your code looks older, it imports _thread instead of threading, but this shouldn't matter here).

Good luck (we should propose a new Github badge: Heisenberg bug hunter ;-)

To be sure not to break your backups you could also start the BiT GUI since it always executes this code path and should trigger the warning in the console (if you already got it before - otherwise you wouldn't have seen the NoneType error):

WARNING: [common/tools.py:2377 __logKeyringWarning] import keyring failed

@pclinuxer
Copy link

pclinuxer commented Oct 13, 2022

Hello:

Exactly like I described it!

Good.

... checked it (using Git master branch - but your code looks older ...

No idea how that could be.
The BiT I have installed proceeds from the Devuan repository which in turn proceeds from the Debian repository.

Back In Time 1.1.24-0.1

If any application from the Debian repository has some systemd nastiness and can work without it, it is then sanitised by the Devuan maintainers and then included in the Devuan repository.

I expect that the version I have installed should be the same one presently available in the Debian Buster (10.4) repository.

Good luck (we should propose a new Github badge: Heisenberg bug hunter ;-)

Heisenbug ...
Don't think it is that sort of bug.

To be sure not to break your backups ...

The curious thing is that BiT is (apparently) working properly.

... you could also start the BiT GUI ...

I think it may be best to let it run in the same conditions the error cropped up in.
It gets me not to be able to find out what is going on.

And I am not fond of the so called harmless notices. 8^/
It is just crud piling up in some dark corner of the code just wating to reach critical mass and screw up everything.

I'll edit the *.py file now, reboot and report back when/if I get another notification.
Lets give it a min of 15 days and a max of 30.
If in four weeks there is no notification, I'll report back.

Thanks for your input.

Best,

PCL

@aryoda
Copy link
Contributor

aryoda commented Oct 13, 2022

@pclinuxer

You are using an old but commonly installed BiT version that is probably based on the Debian package (that is OK so far, even Ubuntu 20.04 is using this version).

I think this a new kind of bug (race-alike condition between package loading and usage; it already happened earlier which I could see in git blame (last change of this code tried to fix a similar problem #473 which introduced this problematic sleep(0.5) call).

@emtiu I would like to keep this bug open, since there is no other way of reproducing it but together with the OP.
I will try to fix this (not high prio since the bug happens only from time to time).
If you assign the issue to me it could help me filtering my issues easier... THX!

Note to self: Add a hint to the warning how to fix this (eg. link to github README)...

@pclinuxer
Copy link

pclinuxer commented Oct 13, 2022

Hello:

@pclinuxer
... using an old but commonly installed BiT version that is probably based on the Debian package ...
It should be based on a Debian package, in this/my case a package from the Debian Buster repository.

This what is listed there:

[backintime-common] (https://packages.debian.org/buster/backintime-common) (1.1.24-0.1)
[backintime-gnome] (https://packages.debian.org/buster/backintime-gnome) (1.1.24-0.1)
[backintime-kde] (https://packages.debian.org/buster/backintime-kde) (1.1.24-0.1)
[backintime-qt4] (https://packages.debian.org/buster/backintime-qt4) (1.1.24-0.1)

There is a newer version in the Debian Bullseye repository (1.2.1-3) and this is reflected in the Devuan Chimaera repository.

I think this a new kind of bug ...
Not Heisenbug? 8^D

Right.
I have made the edition and rebooted the box.
Hopefully I'll be back with no news when 30 days have gone by.

Thanks for taking care of this.

Best,

PCL

@emtiu emtiu added Bug Heisenbug a problem that is not reproducible but random (non-deterministic) and removed Feedback needs user response, may be closed after timeout without a response labels Oct 14, 2022
@pclinuxer
Copy link

pclinuxer commented Oct 26, 2022

Hello:

@aryoda
It was too good to be true. 8^/
Got the dreaded notification in my mbox a while ago:


From groucho@devuan Tue Oct 25 20:15:02 2022
Received: from groucho (uid 1000)
	(envelope-from groucho@devuan)
	id df2cf
	by devuan (DragonFly Mail Agent v0.11);
	Tue, 25 Oct 2022 20:15:02 -0300
From: root (Cron Daemon)
To: groucho
Subject: Cron <groucho@devuan> /usr/bin/nice -n 19 /usr/bin/ionice -c2 -n7 /usr/bin/backintime backup-job >/dev/null
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 8bit
X-Cron-Env: <SHELL=/bin/sh>
X-Cron-Env: <PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin>
X-Cron-Env: <HOME=/home/groucho>
X-Cron-Env: <LOGNAME=groucho>
Date: Tue, 25 Oct 2022 20:15:02 -0300
Message-Id: <63586df6.df2cf.6b4d9b6c@devuan>

Unhandled exception in thread started by <function __log_keyring_warning at 0x7f791b1f00d0>
Traceback (most recent call last):
  File "/usr/share/backintime/common/tools.py", line 1478, in __log_keyring_warning
TypeError: 'NoneType' object is not callable

Previous one was 13 days ago, the only difference being the memory address ie: <function __log_keyring_warning at 0x7f137fbe2488> instead of this latest one where it is <function __log_keyring_warning at 0x7f791b1f00d0>.

Edit: I have gone back to the previouys version of tools.py.

Please advise.

Thanks in advance,

PCL

@aryoda
Copy link
Contributor

aryoda commented Nov 1, 2022

@pclinuxer OK, then undo the manual code change. I will keep this issue in my watch list until I find the time for more experiments on this problem...

@pclinuxer
Copy link

Hello:

... undo the manual code change.
Done.
... in my watch list until I find the time ...
Right.
Let me know if there's something else I can try.

Best,

PCL

@pclinuxer
Copy link

pclinuxer commented Mar 16, 2023

Hello:

I will keep this issue in my watch list until I find the time for more experiments on this problem...

@aryoda: I keep getting the same error in my system mail. As always, completely at random and no discernible way to trigger it.

Today I came across this article from last year:

https://www.freecodecamp.org/news/typeerror-int-object-is-not-callable-how-to-fix-in-python

"... you can avoid the error by not using a built-in function name as a variable identifier and specifying arithmetic signs where necessary."

I cannot make heads or tails from any of that but thought it may be of help to you.
Let me know if you need me to test anything.

Best,

PCL

@aryoda
Copy link
Contributor

aryoda commented Mar 17, 2023

Today I came across this article from last year:

Excellent hint, the article describes "name shadowing" and would explain our bug if
it always occurs for a specific execution path in the code (even though your observation looks "random" but perhaps it isn't).

I see one such possible constellation in the code:

keyring = None
keyring_warn = False
try:
if os.getenv('BIT_USE_KEYRING', 'true') == 'true' and os.geteuid() != 0:
import keyring
from keyring import backend
except:
keyring = None
os.putenv('BIT_USE_KEYRING', 'false')
keyring_warn = True

The variable name keyring is the same as the name of the imported package!

I am currently totally busy and will try to provide an experimental code change (rename to is_keyring_available) when I have more time so that you could give it a try...

I think we even already have a function to check for an available keyring...

@pclinuxer
Copy link

Hello:

Excellent hint, the article describes "name shadowing" and would explain our bug ...

Good!
Finally something to hold on to.

The variable name keyring is the same as the name of the imported package!

As I never got past rudimentary DOS batch files, I'll take your word for it. 8^D

I am currently totally busy and will try to provide an experimental code change (rename to is_keyring_available) when I have more time so that you could give it a try...

No problem, I undertsand.
A bit more time won't change anything, more so taking into account that this bug does not seem to be affecting my back-ups for the time being. I dread having my system/data go south.

I think we even already have a function to check for an available keyring...

Then that would make it a win-win.
Thank you for your input.
I'll be waiting for the code change to test and hopefully solve this.

Thanks in advance.

Best,

PCL

@buhtz buhtz added this to the 1.3.4 milestone Mar 19, 2023
@aryoda
Copy link
Contributor

aryoda commented Jan 4, 2024

I am just preparing a fix for this (solving the name shadowing) and saw two more problems in the code:

  1. Circular dependency of tools.py and logger.py: Each imports the other module

    import logger

    import tools

  2. logger.warning() is called in a new thread but we cannot assume that this function is thread-safe:

    backintime/common/tools.py

    Lines 2744 to 2752 in 1b9e3b3

    def __logKeyringWarning():
    from time import sleep
    sleep(0.1)
    logger.warning('import keyring failed')
    if keyring is None and keyring_warn:
    #delay warning to give logger some time to import
    from threading import Thread
    thread = Thread(target = __logKeyringWarning, args = ())

To get rid of 2. solving 1. is key.

The usual solution for circular dependencies is to refactor the functions of one module (that are called by the other module) into a new 3rd module.

Since logger.py should work as early as possible it should not depend on too many other modules and in fact it uses only one (!!!) single function of tools.py that could be extracted into a new module:

def _do_syslog(message: str, level: int) -> str:
for line in tools.wrapLine(message):
syslog.syslog(level, '{}{}: {}'.format(
SYSLOG_MESSAGE_PREFIX, _level_names[level], line))

tools.wrapLine() is currently only called by one other code location so it is easy to refactor it into a new "stringutils.py" module:

self.menuStatusMessage.setText('\n'.join(tools.wrapLine(message[1],\
size = 80,\
delimiters = '',\
new_line_indicator = '') \
))

@buhtz
Copy link
Member

buhtz commented Jan 4, 2024

I will look deeper into it. I assume that wraplines can be replace by vanilla Python module textwrap.

@aryoda
Copy link
Contributor

aryoda commented Jan 4, 2024

I will look deeper into it. I assume that wraplines can be replace by vanilla Python module textwrap.

Strike, looks very promising!

I will move the old function from tools into logger first and add a TODO comment mentioning textwrap.
Currently I want to avoid changing the well-known output format (may have big impact on the unit tests) and do this separately. Hope this is OK!

@buhtz
Copy link
Member

buhtz commented Jan 4, 2024

Good plan. "Make it so!"

aryoda added a commit to aryoda/backintime that referenced this issue Jan 4, 2024
…s not callable")

* Fix bug: Unhandled exception "TypeError: 'NoneType' object is not callable" in tools.py function __log_keyring_warning (bit-team#820).
   Logging thread removed and logger module correctly initialized as fix. Is "Heisenbug" so 100 % retesting was not possible.
* Refactor: Solved circular dependency between tools.py and logger.py to fix bit-team#820
@aryoda
Copy link
Contributor

aryoda commented Jan 4, 2024

I would like to close this issue after the fix is merged since it is quite lengthy now and the underlying code is almost completely changed by fix so that I think it is better to start a new issue if it should re-occur.

Testing was indeed difficult since it is a "Heisenbug" (sporadic non-reproducible bug)...

Please veto if this issue should be kept open...

@pclinuxer
Copy link

pclinuxer commented Jan 5, 2024

Hello:

Hope you started off 2024 in form. 8^)

On 4 Jan 2024 at 15:22, aryoda wrote:

I would like to close this issue after the fix is merged since it is quite lengthy ...
Indeed it is ...

I had never come across a "Heisenbug" (or had any idea about such a thing) before but due to how it behaves, my guess is that 'lengthy' is just part of the description.

That said, thank you very much for staying on top of it.
I still (sporadically) get notifications for this problem.
ie: BiT 1.1.24-0.1 from the Devuan/ Debian repositories

~$ apt list | grep installed | grep backintime
--- snip ---
backintime-common/oldoldstable,oldoldstable,now 1.1.24-0.1 all [installed]
backintime-qt4/oldoldstable,oldoldstable,now 1.1.24-0.1 all [installed]
~$ 

The very same ones always, never any change but with no discernible parttern but as I have mentioned in my posts, it does not seem (?) to affect BiT peformance but could not swear to that.

... better to start a new issue if it should re-occur.
Please veto if this issue should be kept open...

Not a veto.

But you may want to consider closing issue #820 once the fix is incorporated into the Debian repositories and can be installed via apt update / apt upgrade which is what most (if not all) users will do as part of their 'good practises' routine and I expect would reach the vast majority of Debian/Devuan based distributions.

That way, any issue will by default be against the new fixed version.

Thank you very much for your patience and steady work with this.
Really appreciate it.

Best,

PCL

@aryoda
Copy link
Contributor

aryoda commented Jan 7, 2024

But you may want to consider closing issue #820 once the fix is incorporated into the Debian repositories and can be installed via apt update / apt upgrade which is what most (if not all) users will do as part of their 'good practises' routine and I expect would reach the vast majority of Debian/Devuan based distributions.
[...]
That way, any issue will by default be against the new fixed version.

Our challenge is that we (= BiT developers) do only maintain the source code for the most-recent BiT release (based on our source code) and back porting patches to older BiT versions in distro packages is the job of the package maintainers (in your case Jonathan Wiltshire for Back In Time 1.1.24-0.1 which is quite an old version now - more than 5 years ago).

We have decided a few month ago that an issue will be closed once a fix is merged into our "dev" branch
and the issue is assigned to our the next (source-code-based) release milestone at Github.

Updating distro packages (like for Debian or Devuan) is a downstream process (out of our control and therefore responsibility) so we cannot keep our issues open until each and every distro has incorporated the fixes (and created a new release package).

Our next release that includes this fix is planned for end of January 2024 and there is a chance that updating your BiT installation from our source code release may not work due to newer dependencies that may not be fulfill-able on Devuan/Buster (eg. python3 is still 3.7.3 but BiT requires 3.8 since #1358).

Applying the patch file of my bug fix for v1.2.4 will also not work I guess since I have also changed a renamed file that patch cannot identify.

What I can offer you is to prepare a patch file for the source code of v1.2.4 that you could use for your installation and also to ask the package maintainer and in the corresponding Debian bug report to incorporate this patch.

This would also help to test the fix.

Thank you very much for your patience and steady work with this. Really appreciate it.

Always happy to help (and would have been helpless without your help 😉 )

@buhtz
Copy link
Member

buhtz commented Jan 7, 2024

A distinction must be made between upstream and GNU Linux distributions. Our (upstream) repo including its tickets (Issues & PRs) is focused on upstream and nothing else. It is the policy of the project and do save our spare resources.

If the distro package version is to old you have to open a ticket at your distro (and link it here) and wait for response of the package maintainer or take over the packaging at your distro.
But for this Issue there still is a ticket at Debian (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=990343) and Devuan (https://bugs.devuan.org/cgi/bugreport.cgi?bug=585). Just subscribe to them.

Especially in your case I see you do use "oldoldstable". You can upgrade to "stable" to be a bit more "fresh" with BIT. If you want a distro package with this Issue fixed you need to wait until the next distro release. Debian (and I assume Devuan, too) won't backport such a trivial bug. Maybe you can trigger the creation of a BIT package in the backports repo of Debian/Devuan. Debian do has an automatic system connecting bug tickets to upstream bug tickets and closing them when upstream was fixed and the related upstream version is packaged for Debian.

Not sure why you use "oldoldstable" but asking for backporting a trivial bug fix. I recommend to discuss this topic in your Devuan community to get more insight about how Debian/Devuan do handle such situations.

You can also consider using one of the PPAs. I do not like PPAs, too but sometimes there are reasons.

All this is on the side of us as upstream.

aryoda added a commit that referenced this issue Jan 9, 2024
…allable

* Fix bug: Unhandled exception "TypeError: 'NoneType' object is not callable" in tools.py function __log_keyring_warning (#820).
  Logging thread removed and logger module correctly initialized as fix. Is "Heisenbug" so 100 % retesting was not possible.
* Refactor: Solved circular dependency between tools.py and logger.py to fix #820

Co-authored-by: aryoda <[email protected]>
@aryoda
Copy link
Contributor

aryoda commented Jan 9, 2024

@pclinuxer We are preparing our next release soon (scheduled for mid to end of January).
If you want me to prepare a patch for your BiT version 1.2.4 to "backtest" the fix on your computer please let me know.

@pclinuxer
Copy link

pclinuxer commented Jan 11, 2024

Hello:
@aryoda

... only maintain the source code for the most-recent BiT release ...

Yes, I understand.

... preparing our next release soon ...

Thanks for the offer but I think you have more than enough on your collective plate/s.
It is a very good thing that you seem to have found the issue, that is enough for me.

I reserve a very strong animosity for the often encountered 'won't fix' replies to a bug report.
In my view, a bug is a bug (YMMV), no matter how slight its effect is.

To me, a 'won't fix' reply is the equivalent of sanctioning sloppy/lazy coding and absolutely opposite to the basic Unix/Linux philosophy. ie: dirt swept under the rug that eventually turns into a mole-hill that people will, sooner or later, trip over.

@buhtz

... why you use "oldoldstable" ...

Abandoned applications I am fond of eg: WiCD and for which I have not found a half-decent alternative, just the useless bloat that seems to be the norm these days. SLiM was also on that list but fortunately someone took up the slack and forked it, so now it is alive again.

My ca. 2007 Sun Microsystems Ultra24 WS (Q9550 8Gb) with a three screen setup runs Devuan Beowulf on a backported kernel and a couple of VBox VMs, one headless for PiHole/Unbound DNS.

After few HW upgrades during the second/third year, it is still doing a wonderful job as my daily workhorse.
I guess that I will eventually move on to the next oldstable when the time comes.

... asking for backporting a trivial bug ...

Won't do that, as you say it is trivial and (as far as I have seen) does not affect BiT's performance.

... consider using one of the PPAs.

Not a chance, have to use .appimage files every so often but that is about it.

Once again, thanks for getting to the bottom of the issue.

Best,

PCL

@buhtz
Copy link
Member

buhtz commented Jan 11, 2024

I reserve a very strong animosity for the often encountered 'won't fix' replies to a bug report. In my view, a bug is a bug (YMMV), no matter how slight its effect is.

To me, a 'won't fix' reply is the equivalent of sanctioning sloppy/lazy coding and absolutely opposite to the basic Unix/Linux philosophy. ie: dirt swept under the rug that eventually turns into a mole-hill that people will, sooner or later, trip over.

Kudos to you! ❤️ I strongly agree here. But it is the first time I do read this from someone else.

@pclinuxer
Copy link

Hello:

... strongly agree ...
... first time I do read this from someone else.

Not being a coder/developer/maintainer usually puts me in a difficult position when I speak out about something as obvious as this, but not saying anything only helps to maintain the status quo. ie: only makes things worse.

Thanks for your comment.

Best,

PCL

@pclinuxer
Copy link

pclinuxer commented Jul 1, 2024

Hello:

Just one last word re: #820.

Took a while but I finally upgraded my Devuan Beowulf system (via Chimaera) to Daedalus.
BackInTime from the Devuan repository upgraded to 1.3.3-4.
No issues and working as expected.

Once again, thank you very much for your work.

Best,

PCL

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Heisenbug a problem that is not reproducible but random (non-deterministic)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants