Implement a rqt_bag plotting plugin #239

trainman419 · 2014-06-03T06:17:34Z

Implement the rqt_bag plotting plugin based on the plotting code in rqt_plot, to address #85. I've included the downsampling code originally written by @adwilson10, and upgraded it so that it samples across the selected region in the bag file rather than needing it's own limits, and so that it does the downsampling in a separate thread that doesn't block the main UI thread.

This includes a major overhaul of the plotting code in rqt_plot, to make it useful as a general plotting widget for other plugins.

The overhaul also includes updates to the plotting core to support solutions to #235, #234 and #231.

This also includes an update to the rqt_bag plugin API to expose updates to the selected region as a QT signal, and my initial fix for #67.

I've tested this with a large bag file: 1.9GB, with 500k+ messages on a topic.

This depends on #236; merge that PR first.

DorianScholz · 2014-06-03T09:27:24Z

@trainman419 Thanks for taking on the task of implementing this and the other issues. For the next few month I simply won't have the time to work on this, but I hope I can at least give more feedback. So thanks again for your efforts!

adwilson10 · 2014-06-03T14:08:11Z

@trainman419 Great work! I'll try to spend some time to test it out the next few days, but at a glance it looks like a great improvement in getting all of the backends and threading incorporated.

vrabaud · 2014-06-05T08:55:52Z

Hi @trainman419 , I had ported the old rxbag code to the new API too.
https://github.com/vrabaud/rqt_common_plugins
Maybe there is stuff useful in my code (especially the base class for the plots). Still not finished though (GUI bugs, and not multiple topics yet). The changes also seem less invasive, just let me know if I should stop working on it, thx.

DorianScholz · 2014-06-05T13:13:35Z

I just tested rqt_plot with your changes and it seems to work fine.

I had only minor issues:

output on first curve added:

No plot widget; returning default X limits
No plot widget; returning default Y limits

output on every curve added:

New curve created with no data!

autoscoll checkbox setting was restored, but not respected, on relaunch of rqt_plot
The checkbox showed the correctly restored value (unchecked), but the plot still auto scrolled (in pyqtgraph and matplot) after restarting the plugin.

If 3 is fixed and 1 and 2 are either removed or downgraded to debug output instead of warnings, it would be OK to merge from my side.

trainman419 · 2014-06-05T18:54:46Z

Ok; I've resolved (1) and (3) and removed the warning on (2)

@vrabaud I'll take a look at your port and see what I can use.

adwilson10 · 2014-06-06T15:39:37Z

The only issue I've come across is that the field parsing in lines 247-249 of plot_view.py fails when adding a curve if the field has an array of values, ie the path is data[0]. Replacing those three lines with the following appears to parse correctly as far as I can tell.

y_value = eval('entry[1].' + path)

trainman419 · 2014-06-06T17:32:46Z

@adwilson10 hrm... that does look like a problem. Do you have a solution that doesn't use eval? I prefer not to use eval if it can be avoided.

ablasdel · 2014-06-06T18:22:55Z

+1 Thanks for the in person demo on Thursday btw everything looked awesome!

This one needs a few more days to soak. I will plan to merge on this on Monday (~3 days from this post) if there is no objection.

adwilson10 · 2014-06-07T06:14:22Z

@trainman419 so I think the other way to address arrays is through bracket parsing - replacing line 249 with something like the following appears to work - I've only tested on a 1D array though.

if '[' in field:
    [field, index] = field.split('[')
    y_value = getattr(y_value, field)[int(index.replace(']',''))]
else:
    y_value = getattr(y_value, field)

The UI now appears immediately on startup, but still freezes for a bit _after_ the bag file is done loading and has been added to the timeline. Perhaps this is due to slow rendering?

ablasdel · 2014-06-10T21:06:47Z

Thanks @trainman419 all the work and for this impressive plugin plugin!

Really not regretting leaving that rxbag plugin system in when I ported this over now!

Implement a rqt_bag plotting plugin

trainman419 · 2014-06-10T21:07:36Z

Thanks!

mikepurvis · 2014-07-22T19:47:15Z

Hey all, did this end up in hydro and indigo too?

dirk-thomas · 2014-07-22T19:56:07Z

It is released in Indigo. The repo hasn't been released into Hydro recently and therefore lacks a bit behind. But that will happen in the near future.

http://www.ros.org/debbuild/compare_hydro_indigo.html?q=rqt_common

trainman419 · 2014-07-22T20:23:19Z

+1 to getting this released into Hydro

ablasdel · 2014-07-22T21:08:08Z

+1 for release into Hydro. It was what it was originally tested against so it shouldn't have issue correct?

Changes since 0.2.16: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Changelog for package rqt_reconfigure ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 0.3.8 (2014-07-15) ------------------ 0.3.7 (2014-07-11) ------------------ * fix slider bar, add context menus for common operations (`#251 <https://github.com/ros-visualization/rqt_common_plugins/issues/251>`_) * fix bug in float range calculations (`#241 <https://github.com/ros-visualization/rqt_common_plugins/issues/241>`_) * remove experimental suffix from rqt_reconfigure (`#256 <https://github.com/ros-visualization/rqt_common_plugins/issues/256>`_) * export architecture_independent flag in package.xml (`#254 <https://github.com/ros-visualization/rqt_common_plugins/issues/254>`_) 0.3.6 (2014-06-02) ------------------ * remove unnecessary margins to improve usability on small screens (`#228 <https://github.com/ros-visualization/rqt_common_plugins/issues/228>`_) 0.3.5 (2014-05-07) ------------------ * numerous improvements and bug fixes (`#209 <https://github.com/ros-visualization/rqt_common_plugins/pull/209>`_, `#210 <https://github.com/ros-visualization/rqt_common_plugins/pull/210>`_) * add option to open list of names from command line (`#214 <https://github.com/ros-visualization/rqt_common_plugins/pull/214>`_) 0.3.4 (2014-01-28) ------------------ 0.3.3 (2014-01-08) ------------------ * add groups for rqt plugins, renamed some plugins (`#167 <https://github.com/ros-visualization/rqt_common_plugins/issues/167>`_) * mark rqt_launch and rqt_reconfigure as experimental (`#167 <https://github.com/ros-visualization/rqt_common_plugins/issues/167>`_) 0.3.2 (2013-10-14) ------------------ 0.3.1 (2013-10-09) ------------------ 0.3.0 (2013-08-28) ------------------ * fix updating range limits (`#108 <https://github.com/ros-visualization/rqt_common_plugins/issues/108>`_) * fix layout quirks (`#150 <https://github.com/ros-visualization/rqt_common_plugins/issues/150>`_) * fix icon for closing a node (`#48 <https://github.com/ros-visualization/rqt_common_plugins/issues/48>`_) * fix handling of enum parameters with strings 0.2.17 (2013-07-04) ------------------- * Improvement; "GUI hangs for awhile or completely, when any one of nodes doesn't return any value" (`#81 <https://github.com/ros-visualization/rqt_common_plugins/issues/81>`_) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Changelog for package rqt_topic ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 0.3.8 (2014-07-15) ------------------ 0.3.7 (2014-07-11) ------------------ * export architecture_independent flag in package.xml (`#254 <https://github.com/ros-visualization/rqt_common_plugins/issues/254>`_) 0.3.6 (2014-06-02) ------------------ 0.3.5 (2014-05-07) ------------------ 0.3.4 (2014-01-28) ------------------ 0.3.3 (2014-01-08) ------------------ * add groups for rqt plugins, renamed some plugins (`#167 <https://github.com/ros-visualization/rqt_common_plugins/issues/167>`_) * catch and show exceptions `#198 <https://github.com/ros-visualization/rqt_common_plugins/issues/198>`_ 0.3.2 (2013-10-14) ------------------ 0.3.1 (2013-10-09) ------------------ * improve rqt_topic initialization time (`#62 <https://github.com/ros-visualization/rqt_common_plugins/issues/62>`_) * modified toggling topics to use checkbox instead of context menu (`#75 <https://github.com/ros-visualization/rqt_common_plugins/issues/75>`_) 0.3.0 (2013-08-28) ------------------ * fix cleaning old data in rqt_topic (fix `#74 <https://github.com/ros-visualization/rqt_common_plugins/issues/74>`_) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Changelog for package rqt_top ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 0.3.8 (2014-07-15) ------------------ 0.3.7 (2014-07-11) ------------------ * export architecture_independent flag in package.xml (`#254 <https://github.com/ros-visualization/rqt_common_plugins/issues/254>`_) 0.3.6 (2014-06-02) ------------------ 0.3.5 (2014-05-07) ------------------ 0.3.4 (2014-01-28) ------------------ * fix sort order for numerical fields (`#205 <https://github.com/ros-visualization/rqt_common_plugins/issues/205>`_) 0.3.3 (2014-01-08) ------------------ * add groups for rqt plugins, renamed some plugins (`#167 <https://github.com/ros-visualization/rqt_common_plugins/issues/167>`_) * fix an error caused by SIGKILLing nodes 0.3.2 (2013-10-14) ------------------ 0.3.1 (2013-10-09) ------------------ 0.3.0 (2013-08-28) ------------------ * remove copy of psutil module and implement missing function (`#105 <https://github.com/ros-visualization/rqt_common_plugins/issues/105>`_) 0.2.17 (2013-07-06) ------------------- * Embeds python-psutil in the package in order to be enabled on Ubuntu Precise * first release of this package into hydro ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Changelog for package rqt_image_view ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 0.3.8 (2014-07-15) ------------------ 0.3.7 (2014-07-11) ------------------ 0.3.6 (2014-06-02) ------------------ 0.3.5 (2014-05-07) ------------------ * list image transport topics if parent image topic is not available (`#215 <https://github.com/ros-visualization/rqt_common_plugins/issues/215>`_) 0.3.4 (2014-01-28) ------------------ 0.3.3 (2014-01-08) ------------------ * add groups for rqt plugins, renamed some plugins (`#167 <https://github.com/ros-visualization/rqt_common_plugins/issues/167>`_) * properly handle aligned images * wrap cv calls in try-catch-block (`#201 <https://github.com/ros-visualization/rqt_common_plugins/issues/201>`_) 0.3.2 (2013-10-14) ------------------ 0.3.1 (2013-10-09) ------------------ * fix event handling for rqt_image_view enabling to run multiple instances simultaneously (`#66 <https://github.com/ros-visualization/rqt_common_plugins/issues/66>`_) * add rqt_image_view to global bin (`#168 <https://github.com/ros-visualization/rqt_common_plugins/issues/168>`_) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Changelog for package rqt_bag_plugins ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 0.3.8 (2014-07-15) ------------------ * fix missing installation of resource subfolder 0.3.7 (2014-07-11) ------------------ * add plotting plugin (`#239 <https://github.com/ros-visualization/rqt_common_plugins/issues/239>`_) * fix rqt_bag to plot array members (`#253 <https://github.com/ros-visualization/rqt_common_plugins/issues/253>`_) * export architecture_independent flag in package.xml (`#254 <https://github.com/ros-visualization/rqt_common_plugins/issues/254>`_) 0.3.6 (2014-06-02) ------------------ 0.3.5 (2014-05-07) ------------------ * fix PIL/Pillow error (`#224 <https://github.com/ros-visualization/rqt_common_plugins/issues/224>`_) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Changelog for package rqt_plot ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 0.3.8 (2014-07-15) ------------------ * fix missing installation of Python subpackage 0.3.7 (2014-07-11) ------------------ * fix missing import (`#248 <https://github.com/ros-visualization/rqt_common_plugins/issues/248>`_) * significant improvements and unification of different plot backends (`#239 <https://github.com/ros-visualization/rqt_common_plugins/issues/239>`_, `#231 <https://github.com/ros-visualization/rqt_common_plugins/issues/231>`_) * make more things plottable including arrays and simple message types (`#246 <https://github.com/ros-visualization/rqt_common_plugins/issues/246>`_) * make DataPlot a proxy for its plot widget, redraw after loading new data, add clear_values (`#236 <https://github.com/ros-visualization/rqt_common_plugins/issues/236>`_) * export architecture_independent flag in package.xml (`#254 <https://github.com/ros-visualization/rqt_common_plugins/issues/254>`_) 0.3.6 (2014-06-02) ------------------ * subscribe to any known topic, even if currently not available (`#233 <https://github.com/ros-visualization/rqt_common_plugins/pull/233>`_) 0.3.5 (2014-05-07) ------------------ * change minimum padding to enable viewing arbitrarily small values (`#223 <https://github.com/ros-visualization/rqt_common_plugins/pull/223>`_) * redraw plot only on new data to reduce cpu load, especially with matplot (`#219 <https://github.com/ros-visualization/rqt_common_plugins/issues/219>`_) 0.3.4 (2014-01-28) ------------------ 0.3.3 (2014-01-08) ------------------ * add groups for rqt plugins, renamed some plugins (`#167 <https://github.com/ros-visualization/rqt_common_plugins/issues/167>`_) * add checkbox to toggle automatic scrolling of plot with data * add simple legend for pyqtgraph backend 0.3.2 (2013-10-14) ------------------ 0.3.1 (2013-10-09) ------------------ 0.3.0 (2013-08-28) ------------------ * fix waiting on unpublished topics (`#110 <https://github.com/ros-visualization/rqt_common_plugins/issues/110>`_) * fix rendering of icons on OS X (`ros-visualization/rqt#83 <https://github.com/ros-visualization/rqt/issues/83>`_) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Changelog for package rqt_console ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 0.3.8 (2014-07-15) ------------------ 0.3.7 (2014-07-11) ------------------ * export architecture_independent flag in package.xml (`#254 <https://github.com/ros-visualization/rqt_common_plugins/issues/254>`_) 0.3.6 (2014-06-02) ------------------ 0.3.5 (2014-05-07) ------------------ 0.3.4 (2014-01-28) ------------------ 0.3.3 (2014-01-08) ------------------ * add groups for rqt plugins, renamed some plugins (`#167 <https://github.com/ros-visualization/rqt_common_plugins/issues/167>`_) * use icons instead of text when available, refactor pause/resume button 0.3.2 (2013-10-14) ------------------ 0.3.1 (2013-10-09) ------------------ * rewrite of rqt_console to drastically improve performance (`#186 <https://github.com/ros-visualization/rqt_common_plugins/pull/186>`_) 0.3.0 (2013-08-28) ------------------ * pause button no more saves state (`#125 <https://github.com/ros-visualization/rqt_common_plugins/issues/125>`_) * persist message limit (`#138 <https://github.com/ros-visualization/rqt_common_plugins/issues/138>`_) * add ability to set logger level (`#117 <https://github.com/ros-visualization/rqt_common_plugins/issues/117>`_) * add tooltips to table cells (`#143 <https://github.com/ros-visualization/rqt_common_plugins/issues/143>`_) * improve labels for filters (`#146 <https://github.com/ros-visualization/rqt_common_plugins/issues/146>`_) * fix time column when loading data from file (`#160 <https://github.com/ros-visualization/rqt_common_plugins/issues/160>`_) * fix applying message limit on change (`#133 <https://github.com/ros-visualization/rqt_common_plugins/issues/133>`_) * fix clear button to remove all messages (`#141 <https://github.com/ros-visualization/rqt_common_plugins/issues/141>`_) * fix sorting to use row index to decide order between equal values (except for time column) (`#124 <https://github.com/ros-visualization/rqt_common_plugins/issues/124>`_) * fix locking of message queue * fix rendering of icons on OS X (`ros-visualization/rqt#83 <https://github.com/ros-visualization/rqt/issues/83>`_) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Changelog for package rqt_graph ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 0.3.8 (2014-07-15) ------------------ 0.3.7 (2014-07-11) ------------------ * fix compatibility with Groovy, use TopicStatistics only if available (`#252 <https://github.com/ros-visualization/rqt_common_plugins/issues/252>`_) * export architecture_independent flag in package.xml (`#254 <https://github.com/ros-visualization/rqt_common_plugins/issues/254>`_) 0.3.6 (2014-06-02) ------------------ 0.3.5 (2014-05-07) ------------------ * add displaying of topic/connection statistics along edges (`#214 <https://github.com/ros-visualization/rqt_common_plugins/pull/214>`_) * using CATKIN_ENABLE_TESTING to optionally configure tests (`#220 <https://github.com/ros-visualization/rqt_common_plugins/pull/220>`_) 0.3.4 (2014-01-28) ------------------ 0.3.3 (2014-01-08) ------------------ * add groups for rqt plugins, renamed some plugins (`#167 <https://github.com/ros-visualization/rqt_common_plugins/issues/167>`_) 0.3.2 (2013-10-14) ------------------ 0.3.1 (2013-10-09) ------------------ * modified zooming method to work better on high-res trackpads like Macbook Pros (`#187 <https://github.com/ros-visualization/rqt_common_plugins/pull/187>`_) 0.3.0 (2013-08-28) ------------------ * fix rendering of icons on OS X (`ros-visualization/rqt#83 <https://github.com/ros-visualization/rqt/issues/83>`_) 0.2.17 (2013-07-04) ------------------- * Improve checkbox labels and tooltips wording.

trainman419 mentioned this pull request Jun 6, 2014

rqt_plot loses configuration on startup #229

Closed

trainman419 added 20 commits June 10, 2014 13:40

Start adding plot plugin

beea724

Add new plot plugin to plugins.xml

a0c2254

Add empty but working plot view

c5a55e0

Tinkering with plot plugin

683fce9

Add topic to plugin view constructor

9c18dea

Initialize plot widget on startup

2066f84

Pull in adwilson's matplot plugin

9b35022

Use thread for loading bag files

c2cb201

The UI now appears immediately on startup, but still freezes for a bit _after_ the bag file is done loading and has been added to the timeline. Perhaps this is due to slow rendering?

bag plot mostly not working with rqt_plot backend

7442316

Add vline and limits to matplot and pyqtgraph

9406b9e

Actually use position when drawing vline

32ea817

Unify autoscaling and autoscrolling

963e76a

Make vline persist if a curve is removed

c7c5a5f

Actually use new limits methods

b81f2f2

Fix import after rebase

1f6fd4b

More work on autoscaling

9e672c5

Fix pyqtgraph get limits

b2bec47

Preserve limits when switching backends

8f2b893

Disable autoscrolling

56bd55c

Extract bag and first message on startup

69e69bb

trainman419 added 15 commits June 10, 2014 13:43

Fix bag image viewer

59c5bff

Refactor resampling, add more notes

6612431

Strip data store out of individual plots

c8223f5

Fix limits bug in matplot

cca7851

DataPlot emits a limits_changed event

3dd9a03

Add TODO about pyqtgraph limit setting

1d45e9e

Completely refactor limits and timestep updates

6816df7

Do add_curve and redraw with signals

3872fd8

Actually do resampling in a thread

45c3da5

Update plot on region change, clean up comments

120eec6

Remove verbose warnings

c63631f

Re-add home button

da409ab

Clean up print statements in qwtplot

87c2e90

Fix startup error on plotting

3e3dff2

Clean up and address minor issues

abe95a1

ablasdel pushed a commit that referenced this pull request Jun 10, 2014

Merge pull request #239 from trainman419/rqt_bag_plot

8dca4f0

Implement a rqt_bag plotting plugin

ablasdel merged commit 8dca4f0 into ros-visualization:groovy-devel Jun 10, 2014

This was referenced Jun 10, 2014

rqt_bag_plugins missing plot feature from rxbag_plugins #85

Closed

rqt_bag could display a loading bar for large bag files #67

Closed

trainman419 deleted the rqt_bag_plot branch June 30, 2014 17:29

adwilson10 mentioned this pull request Jul 15, 2014

Add directories to setup.py, CMakeLists.txt #257

Merged

This was referenced Aug 8, 2016

rqt_plot: add missing dependency on numpy #396

Merged

ImportError: No module named numpy #394

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement a rqt_bag plotting plugin #239

Implement a rqt_bag plotting plugin #239

trainman419 commented Jun 3, 2014

DorianScholz commented Jun 3, 2014

adwilson10 commented Jun 3, 2014

vrabaud commented Jun 5, 2014

DorianScholz commented Jun 5, 2014

trainman419 commented Jun 5, 2014

adwilson10 commented Jun 6, 2014

trainman419 commented Jun 6, 2014

ablasdel commented Jun 6, 2014

adwilson10 commented Jun 7, 2014

ablasdel commented Jun 10, 2014

trainman419 commented Jun 10, 2014

mikepurvis commented Jul 22, 2014

dirk-thomas commented Jul 22, 2014

trainman419 commented Jul 22, 2014

ablasdel commented Jul 22, 2014

Implement a rqt_bag plotting plugin #239

Implement a rqt_bag plotting plugin #239

Conversation

trainman419 commented Jun 3, 2014

DorianScholz commented Jun 3, 2014

adwilson10 commented Jun 3, 2014

vrabaud commented Jun 5, 2014

DorianScholz commented Jun 5, 2014

trainman419 commented Jun 5, 2014

adwilson10 commented Jun 6, 2014

trainman419 commented Jun 6, 2014

ablasdel commented Jun 6, 2014

adwilson10 commented Jun 7, 2014

ablasdel commented Jun 10, 2014

trainman419 commented Jun 10, 2014

mikepurvis commented Jul 22, 2014

dirk-thomas commented Jul 22, 2014

trainman419 commented Jul 22, 2014

ablasdel commented Jul 22, 2014