Automated image matching and stitching using matcher-core library #312

jywarren · 2019-06-25T23:14:41Z

For discussion on integration with https://github.com/publiclab/matcher-core/issues/1 !!!

jywarren

Hi! This is really impressive! Congrats!

Just noting that it's really helpful to open a PR as early as possible so that there can be feedback and discussion about the code as it evolves. I know it's helpful to be sort of secluded during tough coding sessions but as soon as possible please do post so we can discuss!

You've done some great work here! I left lots of feedback, and some questions/suggestions about code organization and structure. It'd also be great to see some docs in this PR that explain how to enable this feature in a map - not everyone will want to, so showing how it's supposed to be done helps people make a choice.

Thanks, @rexagod !!!

jywarren · 2019-06-25T23:17:04Z

examples/index.html

+  <script src="../node_modules/leaflet-toolbar/dist/leaflet.toolbar.js"></script>
+  <link href="../node_modules/leaflet-toolbar/dist/leaflet.toolbar.css" rel="stylesheet" />
+  <!-- for pattern-mining utility -->
+  <script src="../node_modules/matcher-core/orb.core.com.js"></script>


Maybe we ought to move this to a unique example file called "matcher"?

@rexagod i think we'd better follow up on this, matcher.html sounds like a good name for the example!

Sounds good, on it! 👍

jywarren · 2019-06-25T23:17:44Z

examples/index.html

+<body>
+  <center>
+    <div class="loader-container">
+      <img width="100" src="leaflet.gif">


So, what is happening while this is showing? The matcher is searching for interest points in some images?

Yup! But now with the caching implemented, I guess we won't be needing this at all from the second iteration onwards, and the initial iteration should take way lesser loading time (~7000 to ~15 frames perf improvement), so I guess it would be better if we should synchronously process those, and remove this loading screen altogether?

jywarren · 2019-06-25T23:19:10Z

src/util/matcher/stitcher.js

+      processedPoints.images[1].getCorner(3).lng - lng_offset
+    ];
+    var zoom_level = map.getZoom();
+    map.setView(overlay.getCenter(), (zoom_level%2?zoom_level+1:zoom_level-1));


I was a bit disoriented when the map itself moved. Maybe it's not necessary?

This actually is something weird. What's happening is that after updating the corners here the changes are actually reflected into the UI after I change the view, maybe there's some sort of cache refresh when map zooms out/in (since the map changes)? This would explain why it's happening. However, I can't yet find a documented way to trigger the same explicity, without the need of changing the view. What do you think?

@rexagod haven't checked this for myself but I think you just need to call overlay._reset()?

actually via #380 setCorners / setCorner methods I exposed should do the trick

jywarren · 2019-06-25T23:20:22Z

src/util/matcher/stitcher.js

+        }
+      }
+    }
+    var best_point =


Could some of this be refactored into a subfunction called something like filterBestPoints() or something? That wouldn't have to mix at all with HTML as this file does?

Hmm. Do you want me to expose an API here, or should this change be incorporated implicitly?

Sure, should I extract this fn into a different file too?

Well if the ability to filter points gets moved back into the core with projector and doesn't mix with the DOM, does that make sense? It could be an exposed API so that the point selection metric could be overridden...

jywarren · 2019-06-25T23:21:11Z

src/util/matcher/stitcher.js

+    var corresponding_best_point =
+      processedPoints.points[1][processedPoints.confidences[0].indexOf(max_)];
+    document.querySelector(
+      "#map > div.leaflet-pane.leaflet-map-pane > div.leaflet-pane.leaflet-marker-pane"


Ah, we should probably not use #map here - different leaflet maps may have different IDs, and also, what if there is more than one Leaflet map per page? We should base this off of the Leaflet map object container!

jywarren · 2019-06-25T23:22:09Z

src/util/matcher/projector.js

+        A = B;
+        idx--;
+      }
+      window.processedPoints = processedPoints;


Could we be passing this around by reference instead of globally?

Whose processedPoints are these?

jywarren · 2019-06-25T23:23:00Z

src/util/matcher/stitcher.js

+    sectors.s01[i] = [];
+    sectors.s10[i] = [];
+    sectors.s11[i] = [];
+    center = processedPoints.images[i].getCenter();


Ah, so shouldn't processedPoints be a property of each image, rather than the other way around?

Ah, i get it. images[0] and images[1] are the two origin images for the match? And is processedPoints actually better named processedPairs? Or matchedPointPairs?

jywarren

And if you could reply to the other comments here too that'd be great! Thanks!!!

jywarren · 2019-07-17T14:00:06Z

src/util/matcher/projector.js

@@ -0,0 +1,86 @@
+function projector(utils, e, array, obj) { // jshint ignore:line


Hi @rexagod could this be abstracted to accept and produce coordinates independently from the Dom? It might be a good thing to add abstractly to the matcher-core lib?

Okay, so just to clarify, you are suggesting that I should include this into the core lib, right? We can definitely do that and expose this into a parameter!

I think that makes sense, as the desire to try to reconcile the two images into the same coordinate system is going to be a pretty common use case!

rexagod · 2019-07-17T16:46:55Z

Sure, @jywarren! Just a question though, should I stall this one a bit and first complete work on the image-sequencer module then start work here, or would you like me to look into this at the same time? I'm fine with whatever you say, so let me know! Thanks!

sashadev-sky · 2019-08-14T04:53:10Z

dist/leaflet.distortableimage.css

@@ -1,3 +1,9 @@
+body {


we have started using more specific css selectors in our library so they don't overwrite anything downstream. see #330

jywarren · 2020-02-08T04:21:35Z

Noting that the promise code noted by @rexagod should now be easier to integrate due to webpack adoption in #512 (comment)

rexagod added 14 commits June 9, 2019 05:17

initiate orb algo UI integration

45ffdee

implement confidence measures into markers

0f17e8a

implement sector density finding method

7613b51

stitcher algorithm implemented -- UI stitching left

afdf20d

auto-stitcher module integrated

04c9411

modules integration into build process

639c25d

add enable/disable functionality

64522b5

re-expose initial APIs

d57a770

stitcher working

60f18ca

implement polylines for top 3 rich matches

71f91e3

add toggle functionality

db66c54

minor refactors

518e7d7

update lockfile

c183696

restructure matcher utils

d4ce53a

jywarren commented Jun 25, 2019

View reviewed changes

rexagod mentioned this pull request Jul 4, 2019

[Google Summer of Code] Second Phase Progress Update #326

Open

rexagod mentioned this pull request Jul 11, 2019

Documentation publiclab/matcher-cli#7

Closed

jywarren mentioned this pull request Jul 15, 2019

New "simple" implementation using matcher-core publiclab/matcher-core#5

Open

jywarren commented Jul 17, 2019

View reviewed changes

rexagod mentioned this pull request Jul 19, 2019

action cable setup publiclab/mapknitter#805

Merged

sashadev-sky reviewed Aug 14, 2019

View reviewed changes

sashadev-sky mentioned this pull request Aug 14, 2019

Allow featureGroup initialization with no overlay corners #381

Merged

5 tasks

sashadev-sky added this to the Version 1.0.0 Release milestone Aug 14, 2019

rexagod self-assigned this Aug 15, 2019

jywarren mentioned this pull request Aug 20, 2019

Add online demo in gh-pages publiclab/matcher-core#14

Closed

jywarren mentioned this pull request Jan 7, 2020

ADD: Webpack module bundler to compile code from /src/ to /dist/ #512

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Automated image matching and stitching using matcher-core library #312

Automated image matching and stitching using matcher-core library #312

jywarren commented Jun 25, 2019

jywarren left a comment

jywarren Jun 25, 2019

jywarren Jul 15, 2019

rexagod Jul 17, 2019

jywarren Jun 25, 2019

rexagod Jul 17, 2019 •

edited

Loading

jywarren Jun 25, 2019

rexagod Jul 17, 2019

sashadev-sky Aug 14, 2019

sashadev-sky Aug 14, 2019

jywarren Jun 25, 2019

rexagod Jul 17, 2019

rexagod Jul 17, 2019

jywarren Jul 19, 2019

jywarren Jun 25, 2019

jywarren Jun 25, 2019

jywarren Jun 25, 2019

jywarren Jun 25, 2019

jywarren Jul 15, 2019

jywarren left a comment

jywarren Jul 17, 2019

rexagod Jul 17, 2019

jywarren Jul 19, 2019

rexagod commented Jul 17, 2019

sashadev-sky Aug 14, 2019

jywarren commented Feb 8, 2020

		@@ -0,0 +1,86 @@
		function projector(utils, e, array, obj) { // jshint ignore:line

Automated image matching and stitching using matcher-core library #312

Are you sure you want to change the base?

Automated image matching and stitching using matcher-core library #312

Conversation

jywarren commented Jun 25, 2019

jywarren left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rexagod Jul 17, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jywarren left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rexagod commented Jul 17, 2019

Choose a reason for hiding this comment

jywarren commented Feb 8, 2020

rexagod Jul 17, 2019 •

edited

Loading