Fix of issue 246 - Make help and helpdesk more robust #688

steffengraber · 2017-03-22T07:54:24Z

With this pull request, NEST uses only Python for the help system if you are on the Python level.
Therefore it doesn‘t matter if it is compiled with MPI or not.

In jupyter notebooks nest.help(obj) open a modal window with the help text.
With nest.help(obj, pager), you can open a texeditor like gedit from a notebook. If you use an terminaleditor (vi, nano, less, …) as pager, again the modal window open up.

If you use nest.helpdesk() in a notebook a new tab is created.

This pull request is a fix for issue #246.

heplesser · 2017-03-23T08:37:44Z

pynest/nest/lib/hl_api_helper.py

+        iptnk = cfg['IPKernelApp']['parent_appname']
+    except NameError:
+        iptnk = ''
+    # test if jupyther notebook


heplesser · 2017-03-23T08:38:25Z

pynest/nest/lib/hl_api_helper.py

@@ -400,6 +404,124 @@ def broadcast(item, length, allowed_types, name="item"):
    return item


+def check_nb():
+    """Check if is notebook or not
+    """


Maybe better

"""Return true if called from a Jupyter notebook."""

heplesser · 2017-03-23T08:46:39Z

pynest/nest/lib/hl_api_helper.py

+        iptnk = ''
+    # test if jupyther notebook
+    jpnk = re.findall(r'.*jupyter.*', os.environ['_'])
+    ipjptk = re.findall(r'.*ipython.*', os.environ['_'])


When calling this from a plain ipython console, it returns a match (Python 2.7.12). So the function would return True also when called from the ipython console, not just from a notebook. Is that intentional?

heplesser · 2017-03-23T08:53:05Z

pynest/nest/lib/hl_api_helper.py

+    elif ipjptk:
+        return True
+    else:
+        return False


You could simplify this to (but see comment above)

return iptnk == 'ipython-notebook' or re.search(r'ipython|jupyter', os.environ['_'])

heplesser · 2017-03-23T08:57:49Z

pynest/nest/lib/hl_api_helper.py

+            });
+        }
+    );
+    """ % (objname, hlptxt)))


% substitution is deprecated in Python 3, .format() should be used instead.

heplesser · 2017-03-23T08:58:54Z

pynest/nest/lib/hl_api_helper.py

+        return False
+
+
+def open_window(objname, hlptxt):


Maybe show_help_in_modal_window() would be a better name? Should this be marked as private with a _?

heplesser · 2017-03-23T08:59:31Z

pynest/nest/lib/hl_api_helper.py

+    """ % (objname, hlptxt)))
+
+
+def pdoc(hlpobj, pager):


Maybe call it show_help_with_pager() would be a better name, and maybe also this one marked private?

heplesser · 2017-03-23T10:35:35Z

@steffengraber I am testing now and here are some experiences:

If NEST_INSTALL_DIR is not defined, nest.help() crashes. It would be better to provide the user with more guidance, something like:

if 'NEST_INSTALL_DIR' not in os.environ:
    print('NEST help needs to know where NEST is installed. Please source nest_vars.sh or defined NEST_INSTALL_DIR manually.')
    return

Instead of os.environ['NEST_INSTALL_DIR'] + "/share/doc/nest/help/" it is better to use os.path.join()
Is it best to use less as default parser, or would should we use the even more default more? no matter what we use as default, we should check that it exists and if not, either use cat or read the help file into Python and print() it.
We should avoid the low-level subprocess.Popen() and rather use the high-level functions. Since we need to maintain compatibility back to 2.7, we should use the Old-style high-level API.
I got this error: FileNotFoundError: [Errno 2] No such file or directory: 'less -S', which is probably because subprocess does not like options combined with the command name in one string (calling from plain python or ipython); the -S comes from .nestrc, if I remove it there, things work
nest.help('abs') works from notebook
`nest.helpdesk() does nothing when called from plain python or ipython or notebook

janhahne

Calling nest.helpdesk() works fine at my workstation. When I call nest.help('iaf_psc_alpha') or any NEST command I end up with the following error

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".../nest-fix246.install/lib64/python2.7/site-packages/nest/lib/hl_api_helper.py", line 234, in stack_checker_func
    return f(*args, **kwargs)
  File ".../nest-fix246.install/lib64/python2.7/site-packages/nest/lib/hl_api_info.py", line 90, in help
    pdoc(hlpobj, pager)
  File ".../nest-fix246.install/lib64/python2.7/site-packages/nest/lib/hl_api_helper.py", line 521, in pdoc
    proc = subprocess.Popen([pager, objf])
  File "/usr/lib64/python2.7/subprocess.py", line 709, in __init__
    errread, errwrite)
  File "/usr/lib64/python2.7/subprocess.py", line 1326, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

Am I using it wrong, is this related to something already mentioned by @heplesser or another bug?

steffengraber · 2017-05-04T06:48:24Z

@janhahne Sorry for the late response.
How do you compile nest? With MPI on or off?
Under which operating system?

janhahne · 2017-05-04T09:50:23Z

@steffengraber I compiled nest with MPI on and my operating system is openSUSE 13.1. The problem still persist with your recent commits. Just to prevent any error on my side: nest.help('iaf_psc_alpha') should be a correct use of nest.help, right?

steffengraber · 2017-05-04T10:07:35Z

@janhahne yes, nest.help('iaf_psc_alpha') is right.
Is python installed, and what version?

janhahne · 2017-05-04T10:33:31Z

@steffengraber I am using python 2.7.6 and usually pynest works fine for me

steffengraber · 2017-05-04T10:58:02Z

@janhahne with Python older than 2.7.8 the help generation did not work correctly. Is it possible for you to update and test ist again?

janhahne · 2017-05-04T11:35:25Z

@steffengraber I updated python to 2.7.13 and did a new clean build, but the error remains the same. I also noticed that if I enter any object that does not exist (as for example nest.help('testtest')) nothing happens, so there is no output and no exception.

steffengraber · 2017-05-04T12:38:45Z

@janhahne Could you please try nest.help('iaf_psc_alpha', 'less').

janhahne · 2017-05-04T21:50:58Z

@steffengraber nest.help('iaf_psc_alpha', 'less') works fine. The corresponding documentation is displayed and no error occurs.

heplesser

It works for me now---thanks! There are just a few little things to clean up, see detailed comments.

heplesser · 2017-05-26T12:29:24Z

pynest/nest/lib/hl_api_info.py

-
-    sr("/helpdesk << /command (%s) >> SetOptions" % browser)
-    sr("helpdesk")
+    url = os.path.join(os.environ['NEST_DOC_DIR'] + "/help", "helpindex.html")


Provide a useful error message if NEST_DOC_DIR is undefined, similar as for missing NEST_INSTALL_DIR.

Here you mix os.path.join() with string addition to build a URL. I think since we want a URL, we should not use os.path.join() here.

heplesser · 2017-05-26T12:46:45Z

pynest/nest/lib/hl_api_info.py

-    sr("helpdesk")
+    url = os.path.join(os.environ['NEST_DOC_DIR'] + "/help", "helpindex.html")
+    if sys.platform[:3] == "win":
+        os.startfile(url)


Why don't you use webbrowser also under Windows?

heplesser · 2017-05-26T12:49:38Z

pynest/nest/lib/hl_api_info.py

+    if sys.platform[:3] == "win":
+        os.startfile(url)
+    if sys.platform[:3] == "dar":
+        webbrowser.get('safari').open_new(url)


Why not webbrowser.open_new(url) also here? Ok, due to this problem in OSX we need to ask for the browser explicitly under macOS are present (10.12.5). Worse, a failing webbrowser.open_new(url) does not even raise a Python exception but returns happily. So I think we need to keep this, but i would be good to document why we do this.

heplesser · 2017-05-26T12:50:44Z

pynest/nest/lib/hl_api_helper.py

+    helpdir = os.path.join(os.environ['NEST_INSTALL_DIR'], "share", "doc",
+                           "nest", "help")
+    objname = hlpobj + '.hlp'
+    # @todo more pager


Isn't the pager list pretty complete?

heplesser · 2017-05-26T12:52:51Z

pynest/nest/lib/hl_api_info.py

-    browser : str, optional
-        Name of the browser to use
+    The system default browser.
+    /helpdesk << /command (firefox) >> SetOptions in ~/.nestrc is now obsolete


I would drop this line of documentation, since it will confuse most users. The nestrc line is still relevant for the SLI level, so I'd leave it in nestrc.

pager list seems to be complete

heplesser · 2017-05-31T12:06:28Z

@steffengraber PEP8 was still not entirely happy:

MSGBLD0195: [PEP8] pynest/nest/lib/hl_api_info.py:74:56: W291 trailing whitespace
MSGBLD0195: [PEP8] pynest/nest/lib/hl_api_info.py:80:59: W291 trailing whitespace

Otherwise, this looks fine to me.

@janhahne Do you also explicitly approve?

steffengraber · 2017-05-31T12:36:43Z

@heplesser
@janhane found 2 more issues I have to address:

exception needed fo something like nest.help('testtest')
nest.help('iaf_psc_alpha', 'less') works perfect for @janhahne but not nest.help('iaf_psc_alpha').

heplesser

I found a few more details ...

heplesser · 2017-06-01T08:05:21Z

pynest/nest/lib/hl_api_helper.py

-                    pager = 'less'
-        rc.close()
-
+        # check if .netsrc exist


heplesser · 2017-06-01T08:06:16Z

pynest/nest/lib/hl_api_helper.py

-        rc.close()
-
+        # check if .netsrc exist
+        rc_file = os.environ['HOME'] + '/.nestrc'


You should use os.path.join(os.environ['HOME'], '.nestrc') for portable path construction; applies in other places as well.

heplesser · 2017-06-01T08:07:01Z

pynest/nest/lib/hl_api_helper.py

+        rc_file = os.environ['HOME'] + '/.nestrc'
+        if os.path.isfile(rc_file):
+            # open ~/.nestrc
+            rc = open(os.environ['HOME'] + '/.nestrc', 'r')


rc = open(rc_file, 'r')

heplesser · 2017-06-01T08:11:07Z

pynest/nest/lib/hl_api_helper.py

+            # open ~/.nestrc
+            rc = open(os.environ['HOME'] + '/.nestrc', 'r')
+            for line in rc:
+                rctst = re.match(r'^\s?%', line)


Could you add a comment explaining/illustrating briefly what this RE looks for; that will make the code more maintainable. Applies also to other REs.

I think it would also be helpful if you could explain what this entire loop is trying to do---I cannot quite get it just from the code.

heplesser · 2017-06-01T08:13:22Z

pynest/nest/lib/hl_api_info.py

-
-    sr("/helpdesk << /command (%s) >> SetOptions" % browser)
-    sr("helpdesk")
+    url = os.path.join(os.environ['NEST_DOC_DIR'] + "/help", "helpindex.html")


Here you mix os.path.join() with string addition to build a URL. I think since we want a URL, we should not use os.path.join() here.

janhahne

@steffengraber Thank you for addressing my issues! Now everything works as it should for me 👍

heplesser · 2017-06-02T11:40:40Z

pynest/nest/lib/hl_api_helper.py

+                        subprocess.call([pager, objf])
+                        fhlp.close()
+                        break
+    if hlperror == True:


if hlperror for tidier code and to make PEP8 happy.

steffengraber · 2017-06-08T04:53:19Z

@heplesser I added some documentation and PEP8 is happy now.

heplesser · 2017-06-09T07:41:36Z

Release notes: Improved help in PyNEST

Help will work even if compiled with MPI
Help will work in Jupyter Notebooks

heplesser

@steffengraber Thanks for the fixes, just two little details about path construction left to make it entirely pythonic and even windows-safe.

heplesser · 2017-06-09T07:52:19Z

pynest/nest/lib/hl_api_info.py


-    sr("/helpdesk << /command (%s) >> SetOptions" % browser)
-    sr("helpdesk")
+    url = os.path.join(os.environ['NEST_DOC_DIR'] + "/help", "helpindex.html")


I still have some issues with this line:

This is not a URL, but the name of a local file to be opened by a browser, so the variable name should be helpfile or similar, not url.

Instead of mixing string concatenation and os.path.join(), just use os.path.join():

helpfile = os.path.join(os.environ['NEST_DOC_DIR'], 'help', 'helpindex.html')

heplesser · 2017-06-09T07:53:28Z

pynest/nest/lib/hl_api_helper.py

+    # reading ~/.nestrc lookink for pager to use.
+    if pager is None:
+        # check if .netsrc exist
+        rc_file = os.environ['HOME'] + '/.nestrc'


This should be

rc_file = os.path.join(os.environ['HOME'], '.nestrc')

Minor Fix.

steffengraber added 9 commits March 17, 2017 13:53

nest.helpdesk() opens now system default browser

bdc87b7

setting and using help pager/viewer

3e33dea

better use in notebook

83346c1

some inline documentation

e7c9f9c

introducing check_nb() and moving to hl_api_helper.py

8a2d69f

introducing modal window in notebook

64f18a5

better help on pure python level

7a3b8f8

minor helptext edits

3edd66e

Merge branch 'master' into fix-issue-246

cdbc3ab

heplesser requested changes Mar 23, 2017

View reviewed changes

Better check for notebooks

6fe70e5

private functions

3e710ae

heplesser assigned steffengraber Mar 24, 2017

heplesser requested a review from janhahne March 24, 2017 14:16

Secure environment setting

fc66fe2

janhahne reviewed Apr 7, 2017

View reviewed changes

change from review

508c008

Right URL for nest.helpdesk()

5d47a80

heplesser requested changes May 26, 2017

View reviewed changes

steffengraber added 4 commits May 31, 2017 13:00

Drop todo

4e124a3

pager list seems to be complete

Insert inline documentation

13caa64

Merge branch 'master' into fix-issue-246

3b5e86d

Set default browser to 'more'

89a5abe

heplesser approved these changes May 31, 2017

View reviewed changes

steffengraber added 3 commits May 31, 2017 15:04

Better .nesrc handling

148fdb5

Better inline documentation

4642c8d

Improvement of calling the help

ee5b97d

heplesser requested changes Jun 1, 2017

View reviewed changes

PEP 8

42da7c3

janhahne approved these changes Jun 2, 2017

View reviewed changes

heplesser reviewed Jun 2, 2017

View reviewed changes

Documentation

b82fa50

heplesser approved these changes Jun 9, 2017

View reviewed changes

heplesser requested changes Jun 9, 2017

View reviewed changes

Removed string concatenation

a86a32c

Minor Fix.

heplesser approved these changes Jun 9, 2017

View reviewed changes

heplesser merged commit d728947 into nest:master Jun 9, 2017

janhahne mentioned this pull request Jun 9, 2017

Error when nest.helpdesk() is called with Python < 2.7.8 #748

Closed

hakonsbm mentioned this pull request Jun 23, 2017

Reset python helper functions to make them work again #766

Merged

heplesser mentioned this pull request Apr 19, 2018

Fix parsing of ~/.nestrc in show_help_with_pager() #931

Closed

steffengraber deleted the fix-issue-246 branch September 6, 2023 05:23

Fix of issue 246 - Make help and helpdesk more robust #688

Fix of issue 246 - Make help and helpdesk more robust #688

Conversation

steffengraber commented Mar 22, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

heplesser commented Mar 23, 2017

janhahne left a comment • edited Loading

Choose a reason for hiding this comment

steffengraber commented May 4, 2017

janhahne commented May 4, 2017

steffengraber commented May 4, 2017

janhahne commented May 4, 2017

steffengraber commented May 4, 2017

janhahne commented May 4, 2017

steffengraber commented May 4, 2017

janhahne commented May 4, 2017

heplesser left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

heplesser commented May 31, 2017

steffengraber commented May 31, 2017

heplesser left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

janhahne left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

steffengraber commented Jun 8, 2017

heplesser commented Jun 9, 2017

heplesser left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

janhahne left a comment •

edited

Loading