docs: design tokens ADR

[adamstankiewicz]

Description

Drafts an ADR about design tokens, style-dictionary, and runtime theming.

Preview

https://github.com/openedx/paragon/blob/005c873e3563389f7dfa02e1414909ce39fac3a4/docs/decisions/0019-scaling-styles-with-design-tokens.rst

Merge Checklist

• If your update includes visual changes, have they been reviewed by a designer? Send them a link to the Netlify deploy preview, if applicable.
• Does your change adhere to the documented style conventions?
• Do any prop types have missing descriptions in the Props API tables in the documentation site (check deploy preview)?
• Were your changes tested using all available themes (see theme switcher in the header of the deploy preview, under the "Settings" icon)?
• Were your changes tested in the example app?
• Is there adequate test coverage for your changes?
• Consider whether this change needs to reviewed/QA'ed for accessibility (a11y). If so, please add wittjeff and adamstankiewicz as reviewers on this PR.

Post-merge Checklist

• Verify your changes were released to NPM at the expected version.
• If you'd like, share your contribution in #show-and-tell.
• 🎉 🙌 Celebrate! Thanks for your contribution.

[netlify]

✅ Deploy Preview for paragon-openedx ready!

Name	Link
🔨 Latest commit	97aaf9b
🔍 Latest deploy log	https://app.netlify.com/sites/paragon-openedx/deploys/64b7e97dbbea7d0008dcc1b4
😎 Deploy Preview	https://deploy-preview-1929--paragon-openedx.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[codecov]

Codecov Report

Patch coverage has no change and project coverage change: +0.64 🎉

Comparison is base (348186d) 90.73% compared to head (97aaf9b) 91.38%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1929      +/-   ##
==========================================
+ Coverage   90.73%   91.38%   +0.64%     
==========================================
  Files         233      234       +1     
  Lines        4082     4157      +75     
  Branches      966     1001      +35     
==========================================
+ Hits         3704     3799      +95     
+ Misses        371      351      -20     
  Partials        7        7              

see 63 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

[bradenmacdonald]

This is super interesting.

I have a few questions:

1. Would you mind giving an example of the contents of a "token file"?
2. Does this only apply to (pre-defined variables) like $size-font-base? e.g. If a theme author needs to inject custom CSS rules like inserting a ::before pseudo-element (assuming that isn't possible with existing $variables), does that part of the theme go into the token file as CSS, or does it need to be put somewhere else?
3. You talk about runtime theming, which is great and something we really want at OpenCraft. But this also sounds like a complex build process (parse>merge>transform>resolve>build>deploy to CDN>apply at runtime). Is it possible to build real-time theming with this, where admins can use a UI to adjust colors and fonts etc. (adjust tokens using a visual editor) and see them applied immediately?

CC @xitij2000

[adamstankiewicz]

Thanks for the questions, @bradenmacdonald (responses inline):

1. Would you mind giving an example of the contents of a "token file"?

I've included more details about different types of tokens, abbreviated examples of JSON snippets, and a link to all the current tokens as they exist currently in the alpha release of design tokens, which we've started to do some QA in a handful of MFEs (starting to figure out a upgrade/migration plan).

2. Does this only apply to (pre-defined variables) like $size-font-base? e.g. If a theme author needs to inject custom CSS rules like inserting a ::before pseudo-element (assuming that isn't possible with existing $variables), does that part of the theme go into the token file as CSS, or does it need to be put somewhere else?

Yes, design tokens are only the "key/value pairs" of a token name (key) to a style property (value). Similar to our SCSS variables today, design tokens themselves don't really define how the resulting CSS variables are used or applied to the theme and UI components via CSS rules. That would come elsewhere, e.g. the core Paragon theme using the resulting output CSS variables or a @edx/brand override as with the existing approach we have today.

On a related note, though, we've often historically treated Paragon's class names (e.g., .pgn-*) as an internal implementation detail that consumers shouldn't necessarily rely on; that way, Paragon has the flexibility to change them as needed without being treated as a breaking change for consumers just because a component's class name changed. This is also an issue in Paragon's current styles approach, irrespective of design tokens.

3. You talk about runtime theming, which is great and something we really want at OpenCraft. But this also sounds like a complex build process (parse>merge>transform>resolve>build>deploy to CDN>apply at runtime). Is it possible to build real-time theming with this, where admins can use a UI to adjust colors and fonts etc. (adjust tokens using a visual editor) and see them applied immediately?

Good distinction. I believe you're getting at the difference between system-wide theming and what I'm calling "organizational" theming in the ADR. I've added some additional thoughts around this distinction to the ADR and some forward thinking (likely out of scope for the initial ADR?) ideas on how to enable broader support for organizational themes.

For what it's worth, in the Enterprise MFEs (frontend-app-admin-portal & frontend-app-learner-portal-enterprise), we do currently support (in an admittedly somewhat hacky way) dynamic user-driven custom themes picked and/or created by Enterprise Admins, so we'll definitely be interested in finding a solution here for this as well.

Let me know if the additions to the ADR helps address these questions 😄

[bradenmacdonald]

@adamstankiewicz Thanks for the excellent additions to this ADR; that helps give me a much clearer picture here and answered most of my questions. 💯

Good distinction. I believe you're getting at the difference between system-wide theming and what I'm calling "organizational" theming in the ADR.

Ah, we actually have both use cases. Even for the "system-wide" case, we'd like it to have a much shorter iteration cycle. Imagine that we host 100 Open edX instances and we want each of those 100 customers to be able to customize their theme in our admin portal. They're each doing "system-wide" customizations for their entire LMS+Studio, but we'd like them to be able to do that theming themselves and see the results immediately. Right now, they can go into our hosting admin panel to choose their colors etc. and we can show a "fake" preview of how it will look (a mock up), but it takes quite some time to actually apply their changes and roll it out to their instance before they can see in their actual LMS / MFE (which we do using simple-theme). Our long-term ambition is to have a theme management tool built into the Open edX platform somewhere that can adjust colors and show the results immediately without a build process.

[adamstankiewicz]

Right now, they can go into our hosting admin panel to choose their colors etc. and we can show a "fake" preview of how it will look (a mock up), but it takes quite some time to actually apply their changes and roll it out to their instance before they can see in their actual LMS / MFE (which we do using simple-theme).

@bradenmacdonald The current process to apply their changes and roll it out to their instance is predominantly slow because of our dependence on SCSS which requires all consuming applications (e.g., 30 MFEs) to get re-built on any theme change.

The intent of this ADR is to take a (large) incremental step towards solving that problem by at least only requiring the theme to re-built rather than all its consuming applications whenever changes are made to the theme by migrating to CSS variables. To keep scope down for the initial release, the goal of this particular ADR is largely around the migration from current state (SCSS) to desired state (CSS variables via design tokens). Doing this incremental step is foundational in that Paragon can support runtime theming at all. That said, we are planning and hoping for this approach to be extended in several ways moving forward in the future, including to achieve the long-term ambition described below:

Our long-term ambition is to have a theme management tool built into the Open edX platform somewhere that can adjust colors and show the results immediately without a build process.

This solution you're describing is one we have at least 3 known use cases for currently, so we'll definitely need to find a generic/flexible solution here for managing the Paragon theme.

My current thinking to enable support for these use cases might be to additionally expose the style-dictionary functionality described in this ADR via a lightweight Node.js Express API that accepts a JSON payload of custom design tokens which will override Paragon's core token values. The response of such an API would be the complete set of Paragon's CSS variables and CSS utility classes re-computed with the custom tokens passed to the API. The CSS output could then be live injected into the HTML document's <head>.

A benefit of this approach is that consumers can take advantage of the style-dictionary capabilities (e.g., the custom color-yiq modifier below determines an appropriate text color given a specified background color to meet the recommended 4.5:1 contrast ratio for a11y).

A secondary benefit is that consumers could also pass their own custom JSON tokens to the Node.JS API (i.e., not overriding core Paragon tokens), and style-dictionary will transform them to CSS variables accordingly. This is shown in the example below where the consumer is overriding Paragon's color.primary.base color token, but then also defining its own color.hero.* tokens as well to generate custom CSS variables (i.e., --color-hero-bg, --color-hero-text, --color-hero-border), by calling a Node.js Express API endpoint /tokens:

// 3 custom brand colors picked by an administrator in a UI
const brandColors = {
  primary: '#BB4430',
  secondary: '#003366',
  tertiary: '#F3E9DC',
};

const response = await fetch('http://localhost:3000/tokens', {
  method: 'POST',
  headers: {
    Accept: 'application/json',
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    color: {
      primary: {
        base: {
          value: brandColors.primary,
        },
      },
      hero: {
        bg: {
          value: brandColors.secondary,
        },
        text: {
          value: '{color.hero.bg}',
          modify: [{ type: 'color-yiq' }],
        },
        border: {
          value: brandColors.tertiary,
        },
      },
    },
  }),
});
const json = await response.json();
console.log(json);

The output of json above would be something like:

{
  "success": true,
  "cssOutput": ":root { /* contains all Paragon's generated CSS variables based on the custom tokens sent in the POST payload above */ }"
}

Then, MFE could use something like react-helmet to inject the cssOutput into the <head> of the HTML document, for example:

<Helmet>
  <style type="text/css">{brandStyles.cssOutput}</style>
</Helmet>

While such a Node.js API that combines custom tokens with Paragon's core tokens to give CSS variables output solves the application of custom theme changes in real time, it does not solve how those custom theme changes would persist (e.g., in a database).

While we have started to think along the lines of the "real time" theming, I'd say it is out of the scope for this initial ADR, but should definitely be on the roadmap. We'd love to collaborate with Open Craft to iterate and build out such a solution for the true real-time / live theming use case, perhaps with the approach described above or otherwise.

cc @xitij2000 @arbrandes @viktorrusakov

[xitij2000]

My current thinking to enable support for these use cases might be to additionally expose the style-dictionary functionality described in this ADR via a lightweight Node.js Express API that accepts a JSON payload of custom design tokens which will override Paragon's core token values. The response of such an API would be the complete set of Paragon's CSS variables and CSS utility classes re-computed with the custom tokens passed to the API. The CSS output could then be live injected into the HTML document's <head>.

I was just replying about below comment, the above was posted :-)

They're each doing "system-wide" customizations for their entire LMS+Studio, but we'd like them to be able to do that theming themselves and see the results immediately. Right now, they can go into our hosting admin panel to choose their colors etc. and we can show a "fake" preview of how it will look (a mock up), but it takes quite some time to actually apply their changes and roll it out to their instance before they can see in their actual LMS / MFE (which we do using simple-theme). Our long-term ambition is to have a theme management tool built into the Open edX platform somewhere that can adjust colors and show the results immediately without a build process.

The build step from a tokens file to built css is under 1 second, and I think for live-previewing I'd like to see if we can improve that by only building a subset i.e. if you're changing variables related to the theme, then only build tokens for that, and patch those variables.

As suggested above, we also have the theme UI output tokens, and when the theme is saved, it can rebuild and redeploy the theme CSS. At that point having a common theme is less important from a real-time theming perspective since only the variables need to be reloaded, and for that we can have a temporary preview mechanism that can use a live-updating theme css used in place of a pre-build variables file. Switching between variable files at runtime can then also enable multiple (light/dark/high-contrast) themes.

[felipemontoya]

nit: there is a typo "dictonary". Also in the next line

[felipemontoya]

@adamstankiewicz thanks a lot for this excelent ADR. I just now learned about the style-dictionary but I can already see how this will have a great impact in getting the first level of MFE customization off the ground. +1 on this decision.

[brian-smith-tcril]

This is super exciting! I'm really looking forward to sinking my teeth into it a bit and getting design tokens up and running on an MFE!

[brian-smith-tcril]

It'd be nice to have an example of organizational theming being done the "anti-pattern way" vs the "design token way" to demonstrate this.

I'm not sure where exactly the specific example could fit into the ADR, but I think this is an incredible illustration of why design tokens/css variables are such a great approach here

[adamstankiewicz]

@brian-smith-tcril I'll give a go at including a brief example of the Enterprise MFEs relying on overriding class names where it would instead be great for those Enterprise MFEs to rely on the design tokens tooling / CSS variables instead for a more complete/holistic organization theming solution 👍

Let me know what you think of the following verbiage I just added to this paragraph:

For example, the Enterprise MFEs within Open edX (e.g., frontend-app-learner-portal-enterprise) inject <style> tag in the HTML document with CSS class name overrides such as .btn-primary { background-color: $enterpriseCustomerPrimaryColor }. Ideally, similar use cases could instead override a CSS variable such as --pgn-color-btn-primary-bg instead such that it gets re-used throughout all Paragon styles, not just .btn-primary.

[brian-smith-tcril]

Part of me feels like it might be better to "rip the band-aid off" and directly use css vars everywhere scss vars were being used. My main worry is having a mix of scss vars and css vars scattered throughout stylesheets preventing a true "source of truth" for a value.

For example, let's say we have my-scss-var-foo set to my-css-var-foo:

• if all the stylesheets use my-scss-var-foo, and never use my-css-var-foo directly, changing my-scss-var-foo to be my-css-var-bar would work as expected.
• howerver, if some stylesheets use my-scss-var-foo and some use my-css-var-foo, changing my-scss-var-foo could lead to unexpected behavior.

I can think of 2 possible ways to prevent this problem:

• Watch PRs that change scss vars like a hawk, don't let it happen
• Only use css vars

I know there was probably a lot of discussion/thought put into this decision, I'd love to hear any thoughts (or if there was a pros/cons list) that came up when building towards this strategy.

[adamstankiewicz]

@brian-smith-tcril I agree, this should likely be revisited. We did "rip the b(r)and-aid off" (pardon the terrible pun with the (r) haha!) in @edx/paragon itself by updating all the general theme and component styles to no longer use the SCSS variable in favor of using the CSS variable directly (e.g., see Alert styles).

However, Paragon also does still have the _variables.scss file around in the alpha release, where each of the SCSS variables point to the associated CSS variable. I agree this isn't ideal for maintainability/understandability (e.g., single source of truth). Would you agree that we should likely delete _variables.scss in the design tokens release, if possible, at this point? Consumers should no longer be importing anything from Paragon's SCSS directly anymore but would need to verify that the build-scss script in Paragon doesn't rely on _variables.scss.

[xitij2000]

OpenCraft have an open source UI for customising themes that currently uses edx/configuration which generates SCSS variables, but we were planning on adapting it in the future to be a standalone MFE for updating the theme using whatever mechanism is adopted for runtime theming.

We'd love to collaborate on this, and potentially make it a kind of first-setup/onboarding MFE.

[adamstankiewicz]

FWIW, I think there's a couple different related use cases here.

As you're describing, I think, there could be a UI for customizing themes/tokens and generating resulting CSS files on disk. I believe there may also be use cases for needing to expose the style-dictionary tooling from Paragon as an API (e.g., on a Node.js Express server) that may be called at runtime to generate CSS variables based on custom token values stored in a database. In this case, since the application needs to generate CSS variables with custom design tokens at runtime, I believe the only way (afaik) to get MFEs to run style-dictionary is through a Node.js-based API.

I will also mention the API-based use case in this paragraph.

[davidjoy]

Enabling such customization to bring comprehensive theming support to micro-frontends is on the roadmap for the Frontend Working Group as a separate initiative outside of Paragon itself.

I'd be happy if this sentence weren't there. 😂 I feel pretty strongly that comprehensive theming is not a thing we want to promote or enable for micro-frontends; we should find less invasive mechanisms for doing common theming tasks.

[adamstankiewicz]

@davidjoy [inform] I updated this sentence to say, "The Frontend Working Group's roadmap includes improvements to micro-frontend customizability.", without an explicit mention of assuming parity to comprehensive theming.

[davidjoy]

[naive question] Is there a way for Figma to be a potential source of truth for the design tokens via some sort of export? Just thinking about designers getting to use a powerful industry standard tool they're familiar with vs. a tool we have to maintain.

[adamstankiewicz]

Yep, this is definitely part of the long term vision! For example, the Tokens Studio plugin for Figma has a GitHub Sync feature that could enable a two-way sync between design tokens defined in code and Figma, where designers could even self-serve update the value of design tokens and see their changes propagate throughout the whole system with minimal engineering effort (e.g., changing a single color across the platform).

I'm surprised this wasn't included in the example potential benefits here! I will include some words about this.

[davidjoy]

Does this ADR encompass a mechanism for runtime updates to the theme? Does that imply that paragon is deployed somewhere and that MFEs reference its deployed stylesheet?

[adamstankiewicz]

Yes, there is a separate chunk of work in frontend-platform (PR) to take the resulting already compiled CSS from Paragon's design tokens and ingest them as external CDN urls into MFEs based on the MFE's configuration.

With this approach, once the configuration is updated (which may be done via runtime MFE config API, without requiring any re-deploys), updates to the Paragon styles could automatically get pulled in.

For initial MVP release, we plan on relying on jsDelivr, an open-source CDN heavily used in the JS ecosystem, to host the @edx/paragon and @edx/brand compiled CSS for MFEs to import.

[davidjoy]

I'm late to the party, admittedly.

There's a lot of good stuff here - what do we think we need to do in order to approve and merge this ADR?

My impression is that there's a lot of forward-looking vision statements in this ADR, I wonder if one path forward is to pare it down to its core that we intend to move forward with now, given that this may be a long, multi-phased effort. It may also be that we need a plan for following up on this ADR and doing the work (apologies for not being up on any prior art there).

Are we able to enumerate the areas of this ADR that still require some discussion?

[felipemontoya]

I was also wondering what was missing for this PR to be approved and merged.
If there is a base that we can agree on to move this forward I'd like to know and help.

We have been building on top of this concept and so far we are happy with the results.
The way we have been using this ADR, together with openedx/frontend-platform#440 is that we build and deploy the MFEs once using paragon alpha and an olive backport of pull/440.

After that we add the css variables file to the mfe_config API.

like so:

"MFE_CONFIG": {
  "PARAGON_THEME_VARIANTS_LIGHT_URL": "https://css-varsify.s3.amazonaws.com/public/0e61a3ff-4ec7-4246-baf2-b9cb8eee6d8d.css"
},

The css variables is built with different strategies. One being paragon. We have also built it with sytle-dict and we even have a django_template for rendering it on the go.

This site only has the learning mfe configured to use paragon-alpha (that's the picture above), but if there is interest in see it in action by someone else, I could build more of the mfes.

[adamstankiewicz]

To all reviewers thus far, I'd love to get the spider webs brushed off this ADR and get it merged into master. Any takers for an approval and/or additional feedback/questions? 👀 On a quick read, I believe everything still (mostly) holds true 😄 cc @davidjoy @xitij2000 @felipemontoya @brian-smith-tcril @arbrandes

[viktorrusakov]

LGTM! We've been working on design tokens for quite a while now and I think @adamstankiewicz does an excellent job summing it all up in this ADR, thank you for doing that! I couldn't have written this better myself, event though I spent a lot of time on making design tokens work in Paragon 🙂

Considering that we are really close to the release of the design tokens work, I really feel like this ADR should finally get merged.

[xitij2000]

Looks good! Really looking forward to being able to build on top of this!
I've played around with the alpha builds and, I think, this summarizes everything well.

Thanks for the work on this!

[edx-semantic-release]

🎉 This PR is included in version 20.45.5 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀