add Chart2Music keyboard and sound accessibility features #6680

aliwelchoo · 2023-07-21T17:53:12Z

Use data and layout to call the chart2music API inside makePlot.

This pull request is for scatter plots only

aliwelchoo · 2023-07-22T11:10:16Z

Closed captions element is required to be visible on the screen to function so need to find a way to integrate it more cleanly
Need to rebase

src/plot_api/accessibility.js

alexcjohnson · 2023-07-24T21:27:25Z

Excited for this!
FYI @julianna-langston @spmealin - I think we can sort out most of what we need but your feedback is of course welcome!

@archmoj thoughts about packaging? Is a 50kb increase in minified bundle acceptable or should we keep chart2music external like MathJax? If we did make it external we'd probably want it included by default in plotly.py output, and we could have Dash detect and autoload it, but plain plotly.js users would need to explicitly load it

archmoj · 2023-07-24T21:36:56Z

package.json

@@ -77,6 +77,7 @@
    "@turf/bbox": "^6.4.0",
    "@turf/centroid": "^6.0.2",
    "canvas-fit": "^1.5.0",
+    "chart2music": "^1.11.0",


Nice MIT library.
To transpile it to es5, please add it to

plotly.js/webpack.config.js

Line 25 in 149c895

include: /node_modules[\\\/](buffer|d3-color|d3-interpolate|is-mobile)[\\\/]/,

Thank you! Knew there'd be something simple I was missing so I could get this into the same style

archmoj · 2023-07-24T21:38:58Z

src/plot_api/accessibility.js

@@ -0,0 +1,71 @@
+"use strict";
+import c2mChart from "chart2music";


To pass current es5 build tests, this file should be written in es5 style similar other files to src.

Do we still need to be writing in es5? Won't webpack take that and give us the desired output? I'd love it if we can start accepting (and eventually converting existing code to) more modern syntax.

For es modules, we have a config rule in place.
I did try to add a rule to transpile this file during build processes; but npm run build wasn't successful in the past hour.
If one could help with setting up the webpack config, it would be great 🙏

Thank you for these tips again! I've made some attempts at this (putting chart2music and/or accessibility.js into the webpack config but struggling with "Cannot find module 'chart2music'" whenever I use require instead of import. I think I'm still missing something

archmoj · 2023-07-24T21:41:33Z

src/plot_api/accessibility.js

+        var {type, x = [], y = [], name = i, text = []} = trace;
+        if(type === 'scatter') {
+            var traceData = [];
+            if ('y' in trace) {


In scatter traces with orientation: 'h', we should use trace.x instead of trace.y.

archmoj · 2023-07-24T21:42:17Z

src/plot_api/accessibility.js

+        }
+        else {
+            // 'Accessibility not implemented for trace type: ' + trace.type
+            return;


Perhaps we should use continue instead of return here.

Yes I think so. There's some overview text c2m reads out when you first focus on the chart, right? Perhaps we can have any skipped traces briefly mentioned there?

I can look into this. Need to look into where that closed captions text element goes in general, not sure about outputting our own messages into it, it's more than just the overview at the start, also reads off the data as you navigate it.

But this general point is a good one, ideally want to be able to support more than just scatter traces and want to support figures containing unsupported traces alongside supported ones.

Then there's also the opportunity for a config option to say which traces should be in accessibility mode

This is an interesting use case. I've created a feature request for chart2music to add config so that you can specify skipped traces (aka unsupported layers). If you think that would work for you, I'll go ahead and get started on a PR for chart2music.

@julianna-langston that would be great! I think that's going to be really important for this project, so that plotly.js and Dash users who care about this can turn on chart2music in all of their charts, knowing that the result will not always be complete but will always be as useful as it can be and will tell them what they're missing. And then over time as both the plotly.js and the chart2music sides add more features, existing applications will automatically improve.

Chart2Music v1.12 now has an unsupported chart type. Examples and documentation

julianna-langston · 2023-07-25T01:04:38Z

src/plot_api/accessibility.js

+
+    c2mChart({
+        title: title_text,
+        type: "line",


If this PR is for scatter plots, you probably want to have type: "scatter".

In the future, if someone wants to support more chart types, chart2music supports an array. So, if you have 2 traces, 1 scatter and 1 line, you could convert that to chart2music's type: ["line", "scatter"]. Just fyi.

Thank you. I think I'll end up extending the scope to as many types as we can

Thank you for supporting our project, @julianna-langston

test/jasmine/assets/create_graph_div.js

marthacryan · 2024-09-25T15:28:51Z

src/accessibility/sonification/index.js

+    var ccElement = initClosedCaptionDiv(gd, c2mContext.ccOptions);
+
+    // TODO:
+    // I believe that title OR title.text could contain the information needed


@archmoj Is that accurate that title or title.text could contain the information needed?

The preferred way of setting a title is via title.text not title.
Setting title to a string is discouraged(& deprecated) but may still widely used in our codes.
We may handle case of title in case title is set to a string instead of an object.

We should drop support for that!

src/accessibility/sonification/index.js

This reverts commit 129eb3b.

archmoj · 2024-10-01T21:01:02Z

src/plot_api/plot_api.js

    );

+    if(gd._context.sonification.enabled) seq.push(sonification.enable_sonification);


IS sonification key always present?

archmoj · 2024-10-01T21:02:51Z

src/sonification/index.js

+    } else {
+        return document.getElementById(config.elId);
+    }


We could simply return and drop else.

Suggested change

} else {

return document.getElementById(config.elId);

}

}

return document.getElementById(config.elId);

archmoj · 2024-10-01T21:03:58Z

src/sonification/xy_scatter_codec.js

+    var x = trace.x && trace.x.length !== 0 ? trace.x : [];
+    var y = trace.y && trace.y.length !== 0 ? trace.y : [];
+
+    for(var p = 0; p < Math.max(x.length, y.length); p++) {


Let's put Math.max(x.length, y.length) in a variable before the loop and use it inside the loop.

archmoj · 2024-10-01T21:05:06Z

src/sonification/xy_scatter_codec.js

+// This codec serves for one x axis and one y axis
+
+function test(trace) {
+    if (!trace || !trace.type || trace.type !== 'scatter') return null;


Does it for scattergl?

marthacryan · 2024-10-04T17:54:40Z

Last commit fixes #7193

Alistair-Welch added 7 commits July 21, 2023 12:43

initial commit, single scatter plot

6fb9c65

single scatter plot

87b4c5c

options and info

64ae505

data and trace labels

d37d24f

false default

d0467bb

remove debugging prints

3a73776

use index if no x provided

b4df959

aliwelchoo changed the title ~~Add Chart2Music keyboard and sound accessibility features~~ add Chart2Music keyboard and sound accessibility features Jul 21, 2023

Alistair-Welch added 5 commits July 22, 2023 09:27

Lib requirement

97b608f

layout destructuring

6205d62

layout defaults

7e40cde

destructure trace

6d9185b

test sourcetype module

935cf54

remove sourcetype

44deae2

archmoj added feature something new community community contribution labels Jul 24, 2023

alexcjohnson reviewed Jul 24, 2023

View reviewed changes

src/plot_api/accessibility.js Outdated Show resolved Hide resolved

aliwelchoo marked this pull request as draft July 24, 2023 21:34

archmoj reviewed Jul 24, 2023

View reviewed changes

julianna-langston reviewed Jul 25, 2023

View reviewed changes

aliwelchoo marked this pull request as ready for review July 26, 2023 16:59

aliwelchoo marked this pull request as draft July 26, 2023 17:00

Alistair-Welch added 3 commits July 28, 2023 08:34

into es5

c38ab67

include mjs files

c6a27be

transpile accessibility

cac174c

marthacryan added 3 commits September 24, 2024 22:06

Remove changes in existing mock

c5ad240

Update baseline

0113d90

Remove function that was introduced through incorrect merge

771029e

marthacryan reviewed Sep 24, 2024

View reviewed changes

test/jasmine/assets/create_graph_div.js Outdated Show resolved Hide resolved

Undo changes to jasmine tests

bfbbbf1

marthacryan reviewed Sep 25, 2024

View reviewed changes

archmoj reviewed Sep 25, 2024

View reviewed changes

src/accessibility/sonification/index.js Outdated Show resolved Hide resolved

marthacryan added 11 commits September 25, 2024 20:55

Revert "Code readability improvements"

cadfd7f

This reverts commit 129eb3b.

Fix syntax

56e582d

remove comment

8e55f98

Remove todo

86d75d8

Schema update

15671b0

Merge branch 'master' of github.com:plotly/plotly.js into chart2music

057e067

Remove accessibility wrapper directory

44f7ad3

Fix filepaths

8e3b62b

Code cleanup

aed1e02

Code cleanup

1777fba

Code cleanup

3a0384f

archmoj reviewed Oct 1, 2024

View reviewed changes

This was referenced Oct 2, 2024

chart2music: auto-generated trace names are incorrect #7193

Open

Chart2Music: when no x or y axis title is present, the title of the chart becomes "Click to enter x axis title" #7194

Open

Chart2Music help menu keybindings are incorrect for mac #7195

Open

archmoj added this to the v3.0.0 milestone Oct 2, 2024

Fix trace bug

dba0b53

archmoj modified the milestones: v3.0.0, v3.1.0 Oct 22, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add Chart2Music keyboard and sound accessibility features #6680

add Chart2Music keyboard and sound accessibility features #6680

aliwelchoo commented Jul 21, 2023

aliwelchoo commented Jul 22, 2023

alexcjohnson commented Jul 24, 2023

archmoj Jul 24, 2023

aliwelchoo Jul 24, 2023

archmoj Jul 24, 2023

alexcjohnson Jul 24, 2023

archmoj Jul 24, 2023

aliwelchoo Jul 30, 2023

archmoj Jul 24, 2023

archmoj Jul 24, 2023

alexcjohnson Jul 24, 2023

aliwelchoo Jul 24, 2023

julianna-langston Jul 25, 2023

alexcjohnson Jul 25, 2023

julianna-langston Aug 28, 2023

julianna-langston Jul 25, 2023 •

edited

Loading

aliwelchoo Jul 25, 2023

Coding-with-Adam Jul 25, 2023

marthacryan Sep 25, 2024

archmoj Sep 25, 2024

marthacryan Sep 25, 2024

archmoj Oct 1, 2024

archmoj Oct 1, 2024

archmoj Oct 1, 2024

archmoj Oct 1, 2024

marthacryan commented Oct 4, 2024

		@@ -0,0 +1,71 @@
		"use strict";
		import c2mChart from "chart2music";

		);

		if(gd._context.sonification.enabled) seq.push(sonification.enable_sonification);

add Chart2Music keyboard and sound accessibility features #6680

Are you sure you want to change the base?

add Chart2Music keyboard and sound accessibility features #6680

Conversation

aliwelchoo commented Jul 21, 2023

aliwelchoo commented Jul 22, 2023

alexcjohnson commented Jul 24, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

julianna-langston Jul 25, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marthacryan commented Oct 4, 2024

julianna-langston Jul 25, 2023 •

edited

Loading