Add greater than symbol to attribute escaping #9963

johngodley · 2018-09-17T13:14:47Z

Although > is a valid attribute character, it is used by wptexturize in WordPress to split HTML tokens. Without escaping wptexturize will incorrectly tokenize a string, causing problems.

For example, add test > thing as the alt tag for an image in Gutenberg, and then view the post:

<img src="kirby.jpg" alt="test > thing" class="wp-image-10"/>

Results in this HTML rendered in WordPress:

<img src="kirby.jpg" alt="test > thing&#8221; class=&#8221;wp-image-10&#8243;/>

Encoding in Gutenberg prevents this problem while being transparent to the Gutenberg visual UI. It does have the effect that if you add a > into a block's attribute in HTML mode it will be replaced by >, and this matches the same behaviour of & being replaced with &.

The problem likely exists for other block types where an attribute is output directly based on user input. For example, custom class names.

Fixes #9915
Fixes #8779

Note this won't fix any existing posts that have a > in an attribute. Re-saving these in Gutenberg should resolve the problem.

How has this been tested?

Additional unit test added for the > symbol, and existing unit tests modified.

It can also be manually verified that adding test > thing to an image's alt tag is displayed as-is in the Gutenberg UI, and results in test > thing appearing in the HTML.

<img src="kirby.jpg" alt="test &gt; thing" class="wp-image-10"/>

Checklist:

My code is tested.
My code follows the WordPress code style.
My code follows the accessibility standards.
My code has proper inline documentation.

jorgefilipecosta

This PR fixed the problem in my tests, and now it looks like we are doing the same the classic editor does 👍
I think we can merge this fix for now, but I wonder if this is not a bug that should be fixed in wptexturize.

aduth · 2018-09-18T16:03:38Z

packages/element/src/serialize.js

@@ -295,13 +306,17 @@ export function escapeLessThan( value ) {
 * "[...] the text cannot contain an ambiguous ampersand [...] must not contain
 * any literal U+0022 QUOTATION MARK characters (")"
 *
+ * Additionally we escape the greater than symbol, as this is used by wptexturize to


As far as @wordpress/element is concerned, it should have no knowledge of a wptexturize existing. Further, there's nothing about this module which implies any relationship to being used in the context of a post (i.e. it can be used to produce HTML which would be injected in an admin screen). These modules are meant to be independently useful without consideration being strongly bound to any one specific context. Holding ourselves honest to this guideline will prevent biasing implementation toward our specific needs.

Now, with that said, it may still be our best option moving forward, though I'm also wondering a few things:

Can this be fixed from wptexturize?

Can this be fixed by running a string replace on the result generated by a serialize call? Or do we need to be granular about the replacement occurring within an attribute value?

And if so, should we consider extension points to our handling of various parts of the serialization? An attribute value visitor, for example?

Considering WordPress' needs as foreign can help surface where these sorts of extension points might be broadly useful.

these modules are meant to be independently useful

I wasn't aware of that, but it's good to know. This did seem like a surely-this-is-too-easy kind of fix and I was expecting it to raise a question or two!

My first thought was to look at wptexturize. It's a monster of an old function and seems complicated to fix without triggering consequences somewhere. Saying that, I'm unfamiliar with the process, and don't know if Gutenberg is going to be tied to a particular base WP version etc. A fix there is certainly ideal, but tackling it in Gutenberg seems more immediately practical.

I then looked at fixing it inside the blocks themselves, but this places the burden on the block creator.

I looked at the serialize code. I did consider doing a string replace, but fixing a regex bug with another regex seemed like double trouble! It should be possible though, and we don't need to be granular about it - any unencoded > in an attribute will break the post. This could leave the core Gutenberg code to produce clean HTML, and we then have a 'texturizing' layer.

I also looked further into the serialization code and wondered if we could do it at a higher level - somehow traverse the block output and add the extra encoding to attributes. It got complicated, and I then came across escapeAttribute, which as solutions go seemed ideally magical.

By extension points do you mean hooks? That would work, and we could target different environments. If wptexturize is fixed in a future WordPress, the > encoding could be removed etc

By extension points do you mean hooks? That would work, and we could target different environments. If wptexturize is fixed in a future WordPress, the > encoding could be removed etc

In my mind I had pictured something like an AST / other tree visitor, which could be passed as an option to serialize, or in the creation of a custom serializer (e.g. expose serialize factory method to create / override default behavior).

Prior art:

https://github.com/ctrlplusb/react-tree-walker#visitor

https://github.com/jamiebuilds/babel-handbook/blob/master/translations/en/plugin-handbook.md#toc-visitors

@WordPress/gutenberg-core I know there is a core ticket for this https://core.trac.wordpress.org/ticket/45387 It doesn't seem like it has progressed much. Shall we land this fix as a preventive measure while the core fix is ongoing?

These modules are meant to be independently useful without consideration being strongly bound to any one specific context.

Using > in an attribute is valid HTML. Using > in an attribute is also valid HTML. Important to this discussion: the two have the same meaning.

If we're aiming to be as general as possible, we should pick > since it's useable in more contexts.

Using > in an attribute is valid HTML. Using > in an attribute is also valid HTML. Important to this discussion: the two have the same meaning.

I don't disagree, but:

The expressed intention of this function was to apply the minimal transformations necessarily to transform an attribute value to be conformant as per the specification of what constitutes a valid attribute value. Exceptions muddy expectations, both for the consumer and for the future maintainer.

The bug still remains in WordPress that valid HTML becomes mangled upon save (whether from Gutenberg or elsewhere), but because there are few who feel comfortable touching the relevant texturize code, the impetus has been placed on integrators to implement workarounds, as we've done here.

None of this matters, as the pull request has been approved on the pragmatic consideration that this be solved.

You're right. Thank you for your patience and for humoring me :)

I wanted to use this package elsewhere and found I could not because of the details of its implementation. In attempting to understand why, I found this ticket. My comment was meant to be a springboard into that discussion elsewhere, but I instead derailed the conversation. My apologies.

@mdawaffe Not at all derailed, certainly not. I think I'd stressed far too much the idea of being "generic" in my original comment, and it was a good prompt to reevaluate what I'd meant in terms of the stated purpose of a module / function.

johngodley · 2018-11-23T09:30:18Z

Thanks for the nudge @nosolosw !

If the core ticket gets any movement then that may ultimately be the preferred route.

If not then based on Andrew's feedback it seems like a more comprehensive solution is preferred in the long run.

However, if this PR makes sense for the immediate future then I've rebased and fixed it up so it's working again. I can also create another ticket with the desired plans.

ellatrix · 2018-11-26T13:58:10Z

packages/escape-html/src/index.js

@@ -64,7 +75,7 @@ export function escapeLessThan( value ) {
 * @return {string} Escaped attribute value.
 */
 export function escapeAttribute( value ) {
-	return escapeQuotationMark( escapeAmpersand( value ) );
+	return escapeGreaterThan( escapeQuotationMark( escapeAmpersand( value ) ) );


Could use a comment explaining why it diverges from HTML spec.

Ah yes, I think it got lost in the rebase. Added!

packages/escape-html/src/index.js

aduth · 2018-11-30T14:10:22Z

packages/escape-html/src/index.js

+ *
+ * @return {string} Escaped string.
+ */
+export function escapeGreaterThan( value ) {


I think it would be best if we avoided exposing this on the public interface of the module, to avoid committing ourselves to future maintenance if it's arguably a workaround for a WordPress quirk.

Noting that it's exported for use by testing, alternatives could include one of either:

Moving it to a separate file from which it is the default export, but is imported and then not subsequently exported by index.js

Naming it as an unstable API __unstableEscapeGreaterThan

ellatrix · 2018-12-04T11:05:31Z

packages/escape-html/src/index.js

+ *
+ * @return {string} Escaped string.
+ */
+export function __unstableEscapeGreaterThan( value ) {


Why are we even exposing it? If you're exposing it for tests, I'd recommend making a separate file and not export it in index.js.

Related: #9963 (comment)

mcsf · 2018-12-09T16:31:45Z

Punting, as we seemingly haven't reached consensus yet.

gziolo · 2019-01-31T18:22:25Z

Punting, as we seemingly haven't reached consensus yet.

What are next steps? Should we land it or close it? What about the issues that are opened and this PR tries to resolve?

johngodley · 2019-01-31T18:28:22Z

What are next steps? Should we land it or close it?

I have some changes to address the points about exposing unstable code. Once I've got my e2e tests working properly I'll add the changes and if it's still useful then hopefully it will push the PR forward.

johngodley · 2019-02-01T09:21:46Z

Ok, I’ve updated this so the unstable code is also in a separate file. I’ve left the tests in the same place as they seem to make more sense there. I’ve also included changes for the package version and history (pending a real date)

codecov · 2019-03-14T14:15:44Z

Codecov Report

❗ No coverage uploaded for pull request base (master@589a634). Click here to learn what that means.
The diff coverage is 100%.

@@            Coverage Diff            @@
##             master    #9963   +/-   ##
=========================================
  Coverage          ?   48.76%           
=========================================
  Files             ?      612           
  Lines             ?    13641           
  Branches          ?     2541           
=========================================
  Hits              ?     6652           
  Misses            ?     5917           
  Partials          ?     1072

Impacted Files	Coverage Δ
packages/element/src/serialize.js	`100% <100%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 589a634...7892dba. Read the comment docs.

ellatrix · 2019-03-14T14:59:02Z

Rebased this PR, but I have no idea why the tests are failing...

westonruter · 2019-05-14T17:51:44Z

Has the code here been reverted in any way? When adding > or < into an attribute value, the serializer forces it to be > and < respectively.

<!-- wp:paragraph -->
<p><abbr title="1 > 2">False</abbr> and <abbr title="1 < 2">True</abbr></p>
<!-- /wp:paragraph -->

This gets rendered by WordPress as:

<p><abbr title="1 > 2&#8243;>False</abbr> and <abbr title="1 < 2">True</abbr></p>

The “False and” text is erroneously consumed be the attribute:

In this case, it seems to be a problem with wptexturize(), as it is converting the attribute closing " into ″.

But if the block serializer output > as > then this symptom could be avoided.

aduth · 2019-05-14T18:10:51Z

@westonruter Nothing more recent has occurred here to my knowledge. Could you create an issue which includes version information and sample code? I might wonder how your abbr elements are generated could have an impact on the serialization.

westonruter · 2019-05-14T18:25:29Z

@aduth cheers. Added here: #15636.

johngodley added the [Package] HTML entities /packages/html-entities label Sep 17, 2018

Soean added the [Type] Bug An existing feature does not function as intended label Sep 17, 2018

jorgefilipecosta approved these changes Sep 17, 2018

View reviewed changes

aduth reviewed Sep 18, 2018

View reviewed changes

mtias added this to the 4.6 milestone Nov 22, 2018

johngodley force-pushed the fix/texturize-encoding branch from d675a2b to 3d339ca Compare November 23, 2018 09:15

ellatrix reviewed Nov 26, 2018

View reviewed changes

mtias modified the milestones: 4.6, 4.7 Nov 29, 2018

youknowriad requested a review from aduth November 30, 2018 08:12

aduth previously requested changes Nov 30, 2018

View reviewed changes

ellatrix reviewed Dec 4, 2018

View reviewed changes

mcsf modified the milestones: 4.7, 4.8 Dec 9, 2018

aduth mentioned this pull request Dec 11, 2018

Generated aria labels for links in block elements can destroy the output #12683

Closed

youknowriad modified the milestones: 4.8, 4.9 Dec 19, 2018

jorgefilipecosta modified the milestones: 4.9 (Gutenberg), 5.0 (Gutenberg) Jan 18, 2019

johngodley force-pushed the fix/texturize-encoding branch from 350e10a to 06f9027 Compare February 1, 2019 09:05

gziolo modified the milestones: 5.0 (Gutenberg), 5.1 (Gutenberg) Feb 1, 2019

noisysocks modified the milestones: 5.1 (Gutenberg), 5.2 (Gutenberg) Feb 18, 2019

gziolo removed this from the 5.2 (Gutenberg) milestone Mar 4, 2019

youknowriad removed this from the 5.3 (Gutenberg) milestone Mar 18, 2019

Update the docs

7f6b6d2

youknowriad added this to the 5.3 (Gutenberg) milestone Mar 18, 2019

youknowriad merged commit c7801a8 into master Mar 18, 2019

youknowriad deleted the fix/texturize-encoding branch March 18, 2019 07:49

This was referenced Mar 18, 2019

Bump plugin version to 5.3.0-rc.1 #14489

Merged

Update plugin version to 5.3 #14523

Merged

Update plugin version to 5.3 #14524

Closed

Update WordPress trunk to Gutenberg 5.3 #14532

Closed

youknowriad pushed a commit that referenced this pull request Mar 20, 2019

Add greater than symbol to attribute escaping (#9963)

6e18f2f

youknowriad pushed a commit that referenced this pull request Mar 20, 2019

Add greater than symbol to attribute escaping (#9963)

4abfd11

westonruter mentioned this pull request May 13, 2019

Unexpected AMP validation error when aria-label contains > ampproject/amp-wp#2291

Closed

westonruter mentioned this pull request May 14, 2019

Characters for greater than and less than are not encoded as entities in attribute values #15636

Open

snyk-bot mentioned this pull request Nov 18, 2020

[Snyk] Fix for 1 vulnerabilities wpscholar/gutenberg#12

Open

snyk-bot mentioned this pull request Dec 17, 2020

[Snyk] Security upgrade jest from 25.5.4 to 26.0.0 wpscholar/gutenberg#14

Open

This was referenced Mar 30, 2021

[Snyk] Security upgrade @jest/reporters from 25.5.1 to 26.0.0 wpscholar/gutenberg#23

Open

[Snyk] Fix for 1 vulnerabilities wpscholar/gutenberg#24

Open

This was referenced Sep 15, 2021

[Snyk] Fix for 1 vulnerabilities Verdinjoshua26/gutenberg#20

Open

[Snyk] Security upgrade @jest/reporters from 25.5.1 to 26.0.0 Verdinjoshua26/gutenberg#22

Open

Alex-Hay mentioned this pull request Jun 2, 2023

Great than character in theme template causing issues Log1x/acf-composer#160

Closed

This was referenced Jun 23, 2023

[Snyk] Fix for 1 vulnerabilities Verdinjoshua26/gutenberg#77

Open

[Snyk] Fix for 1 vulnerabilities Verdinjoshua26/gutenberg#79

Open

Verdinjoshua26 mentioned this pull request Sep 7, 2023

[Snyk] Security upgrade jest from 25.5.4 to 26.0.0 Verdinjoshua26/gutenberg#85

Open

mcsf mentioned this pull request May 27, 2024

Is the "<" or the ">" character always escaped in the database? Zodiac1978/wp-search-ignore-block-names#17

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add greater than symbol to attribute escaping #9963

Add greater than symbol to attribute escaping #9963

johngodley commented Sep 17, 2018

jorgefilipecosta left a comment •

edited

Loading

aduth Sep 18, 2018

johngodley Sep 19, 2018

aduth Oct 12, 2018

oandregal Nov 22, 2018

mdawaffe Mar 14, 2019

aduth Mar 14, 2019 •

edited

Loading

mdawaffe Mar 14, 2019 •

edited

Loading

aduth Mar 14, 2019

johngodley commented Nov 23, 2018

ellatrix Nov 26, 2018

johngodley Nov 26, 2018

aduth Nov 30, 2018

ellatrix Dec 4, 2018

aduth Dec 4, 2018

mcsf commented Dec 9, 2018

gziolo commented Jan 31, 2019 •

edited

Loading

johngodley commented Jan 31, 2019 •

edited

Loading

johngodley commented Feb 1, 2019

codecov bot commented Mar 14, 2019 •

edited

Loading

ellatrix commented Mar 14, 2019

westonruter commented May 14, 2019

aduth commented May 14, 2019

westonruter commented May 14, 2019

Add greater than symbol to attribute escaping #9963

Add greater than symbol to attribute escaping #9963

Conversation

johngodley commented Sep 17, 2018

How has this been tested?

Checklist:

jorgefilipecosta left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aduth Mar 14, 2019 • edited Loading

Choose a reason for hiding this comment

mdawaffe Mar 14, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

johngodley commented Nov 23, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mcsf commented Dec 9, 2018

gziolo commented Jan 31, 2019 • edited Loading

johngodley commented Jan 31, 2019 • edited Loading

johngodley commented Feb 1, 2019

codecov bot commented Mar 14, 2019 • edited Loading

Codecov Report

ellatrix commented Mar 14, 2019

westonruter commented May 14, 2019

aduth commented May 14, 2019

westonruter commented May 14, 2019

jorgefilipecosta left a comment •

edited

Loading

aduth Mar 14, 2019 •

edited

Loading

mdawaffe Mar 14, 2019 •

edited

Loading

gziolo commented Jan 31, 2019 •

edited

Loading

johngodley commented Jan 31, 2019 •

edited

Loading

codecov bot commented Mar 14, 2019 •

edited

Loading