Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DESIGN: Add alternative designs to the design doc on function composition #806

Merged
merged 7 commits into from
Oct 14, 2024

Conversation

catamorphism
Copy link
Collaborator

@catamorphism catamorphism commented Jun 4, 2024

This document sketches out some alternatives for the machinery provided to enable function composition.

The goal is to provide an exhaustive list of alternatives.

@catamorphism catamorphism changed the title [DRAFT, don't read yet] DESIGN: Add a sequel to the design doc on function composition DESIGN: Add a sequel to the design doc on function composition Jun 4, 2024
@catamorphism catamorphism marked this pull request as ready for review June 4, 2024 13:50
@catamorphism
Copy link
Collaborator Author

Note: this is still pretty rough, but my goal in submitting this as a PR is to check whether the list of alternatives presented is complete. I wanted to get feedback on that before finishing the doc.

Copy link
Collaborator

@eemeli eemeli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am uncomfortable with the further fracturing of this conversation across yet another document, but I guess this is how we're doing this?

I didn't pay particular attention to it before, but indeed the "part 1" of this explicitly notes that it is not going to evolve towards presenting any actual design. Will this document do that, or is it the intent that this too will be just setup for a final design to be presented in the data flow design doc?

I hope that's not the case, and that this document will evolve into a conclusion for this discussion, and that the earlier dataflow document can be dismissed. Otherwise, I would expect for each of these three documents to undergo significant edit work, so that they're significantly condensed. Already now they contain more text than our spec's syntax and formatting sections put together, and that seems disproportionate.

@catamorphism
Copy link
Collaborator Author

I am uncomfortable with the further fracturing of this conversation across yet another document, but I guess this is how we're doing this?

That's a fair point. There was even more that I was going to put into part 2, but I decided to leave everything out except the list of alternatives. Given that, it makes more sense to combine it with part 1, which I've done in a subsequent patch.

I didn't pay particular attention to it before, but indeed the "part 1" of this explicitly notes that it is not going to evolve towards presenting any actual design. Will this document do that, or is it the intent that this too will be just setup for a final design to be presented in the data flow design doc?

The document now presents several designs (with the changes in this PR) and I revised the introduction accordingly.

I hope that's not the case, and that this document will evolve into a conclusion for this discussion, and that the earlier dataflow document can be dismissed. Otherwise, I would expect for each of these three documents to undergo significant edit work, so that they're significantly condensed. Already now they contain more text than our spec's syntax and formatting sections put together, and that seems disproportionate.

I agree that the document needs some more edit work. Potentially the earlier dataflow document could be abandoned. Before I do that, I want to make sure I've identified the full set of alternative designs; the goal of this PR is to provoke that discussion.

I also agree with your observation about the size of these documents compared to the spec itself. I don't see a way around that, though. The alternative is to keep the spec as it is, while allowing the behavior of the use cases in the doc to be implementation-specific. If we do want these use cases to be portable, then I think the complexity is innate. I'm open to any suggestions for making it more concise.

@catamorphism catamorphism changed the title DESIGN: Add a sequel to the design doc on function composition DESIGN: Add alternative designs to the design doc on function composition Jun 5, 2024
@catamorphism
Copy link
Collaborator Author

For those who haven't read through the whole document before, I suggest just reading the diff and commenting if anything in it is unclear. I'll add cross-references accordingly.

@catamorphism catamorphism requested a review from stasm June 5, 2024 11:23
@aphillips aphillips added design Design principles, decisions normative formatting LDML46 LDML46 Release (Tech Preview - October 2024) labels Jun 5, 2024
Copy link
Collaborator

@eemeli eemeli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: this is still pretty rough, but my goal in submitting this as a PR is to check whether the list of alternatives presented is complete. I wanted to get feedback on that before finishing the doc.

This is missing the interface presented in #728. The Preservation Model is pretty close, but it includes ValueType and getInput() definitions that I think are unnecessary.

Comment on lines +989 to +998
* `ValueType`: This type is the union of an `InputType` and a `MessageValue`.

It's tagged with a string tag so functions can do type checks.

```
interface ValueType {
type(): string
value(): unknown
}
```
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From context I gather that the "It" that is tagged is ValueType, of which an interface definition is shown. But how can this be a "union of an InputType and a MessageValue" if the former is wholly implementation-defined? For example in a JS context supporting formatted parts, InputType will likely need to be something like the TS any, as all values could conceivably pass through the formatter.

Also, the mixing of "Type" and "Value" suffixes is confusing. Why not InputValue instead of InputType, and e.g. AnyValue instead of ValueType?

interface MessageValue {
formatToString(): string
formatToX(): X // where X is an implementation-defined type
getInput(): ValueType
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this need a getInput()? Text below mentions that "This solution allows functions to pipeline input,
operate on output, or both", but I don't understand what pipelining input here means, or why the consumer of a MessageValue would need to have access to its input.

and for implementors to document that information
for users.

### Allow both kinds of composition (with different syntax)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With this alternative, can an annotated value be used as an option value? If so, how does that work?

Can an :integer be "pipelined" into a :number, or a :datetime into a :date?

Comment on lines +1054 to +1056
The [function registry data model](https://github.com/unicode-org/message-format-wg/blob/main/spec/registry.md)
could be extended to define `Number` and `String`
as subtypes of `MessageValue`.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be best then to name the types as e.g. MessageNumber and MessageString, as the generic names don't make it clear that these types have MF2-specific behaviours attached to them, such as formatting and selection.

I'm also not really sure how this is different from the two subsequent alternatives, except that the methods of a MessageValue are less explicitly defined.

@catamorphism catamorphism marked this pull request as draft June 10, 2024 08:12
@catamorphism
Copy link
Collaborator Author

I'm converting this back to draft mode. I'm experimenting with the value type in my implementation and I want to finish that before revising this text.

@mihnita mihnita added the blocker-candidate The submitter thinks this might be a block for the Technology Preview label Aug 28, 2024
@aphillips aphillips removed the LDML46 LDML46 Release (Tech Preview - October 2024) label Sep 17, 2024
@aphillips aphillips added the LDML46.1 MF2.0 Draft Candidate label Sep 17, 2024
@catamorphism
Copy link
Collaborator Author

I'm inclined to close this PR because I don't think there's anything left that's not addressed by #728 -- any objections?

@aphillips
Copy link
Member

I have no objections personally, although we might want to capture the materials (it is a design doc, after all), perhaps with the status "rejected" or "superseded" or "obsolete"?

@aphillips aphillips added resolve-candidate This issue appears to have been answered or resolved, and may be closed soon. Agenda+ Requested for upcoming teleconference and removed blocker-candidate The submitter thinks this might be a block for the Technology Preview labels Oct 11, 2024
@catamorphism
Copy link
Collaborator Author

I have no objections personally, although we might want to capture the materials (it is a design doc, after all), perhaps with the status "rejected" or "superseded" or "obsolete"?

I'm fine with adding "obsolete" to the original doc, but the original doc is already in the repo. This PR just adds more content to the doc. So the question is whether it's useful content or not.

@aphillips
Copy link
Member

It's a lot of material to drop on the floor. I added it to the agenda for a couple minutes of discussion.

@aphillips aphillips marked this pull request as ready for review October 14, 2024 18:43
Copy link
Member

@aphillips aphillips left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the 2024-10-14 call we agreed to merge this material and then mark the design document as obsolete. Approving its merger. I will adjust the page status manually.

<dt>Contributors</dt>
<dd>@catamorphism</dd>
<dt>First proposed</dt>
<dd>2024-06-xx</dd>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
<dd>2024-06-xx</dd>
<dd>2024-06-04</dd>

Comment on lines 58 to 62
Not addressed here: the behavior of compositions of built-in functions
(but the choice here will determine what behaviors are possible).

Not addressed here: the behavior of compositions of custom functions
(which is up to the custom function implementor).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are probably invaluable as examples, though. This stuff is difficult to talk about in the abstract.

Comment on lines 66 to 67
A message that has a valid result in one implementation
should not result in an error in a different implementation.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is good, but probably insufficient?

  • A declaration's side effects should be consistent across implementations.
  • Option passing should be consistent between implementations

I think there is a hidden assumption in your requirement (the function/option set must be implemented and installed in both implementations)

Comment on lines +1039 to +1051
.local $age = {$person :getAge}
.local $y = {$age :duration skeleton=yM}
.local $z = {$y :uppercase}
```

In an informal notation,
the three custom functions in this example
have the following type signatures:

```
getAge : Person -> Number
duration : Number -> String
uppercase : String -> String
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I agree with your implied logic here.

The getAge function doesn't change $person to a number. It changes (assigns) $age a number value.

The duration function doesn't change $age to a string. It assigns $y a value. In your example, $y is a string, but that might be inconsistent with what a duration formatter does. It's probably better here to use familiar functions, whose operation we know. I would probably replace getAge with getBirthDate and use date or datetime as the function here. If we did that... I'm not sure that $y is a string. I think then that $y is a an annotated date (and the reason for annotation would be to "float" the birthdate to prevent local time zones from changing the value--as well as preparing the value to be formatted later).

That leaves $z, which presumably calls $y's annotation to get a string, or, resolves the value of $y. If $y is still a date, its string representation might be the potentially-surprising ISO format (2024-06-05T00:00:00)

Comment on lines +1022 to +1035
Types are a way for users of a language
to reason about the kinds of data
that functions can operate on.
The most ambitious solution is to specify
a type system for MessageFormat functions.

In this solution, `ValueType` is not what is defined above,
but instead is the most general type
in a system of user-defined types.
(The internal definitions are omitted.)
Using the function registry,
each custom function could declare its own argument type
and result type.
This does not imply the existence of any static typechecking.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have carefully avoided types so far, although we have to some degree handwaved about types in the registry. To wit:

The operand of a number function is either an implementation-defined type or a literal whose contents match the number-literal production in the ABNF. All other values produce a Bad Operand error.

I think a better way to think about this solution is to separate the operand and the resolved value of an annotation. The operand isn't changed by the function, but the resolved value potentially is.

@aphillips aphillips merged commit f6b724b into unicode-org:main Oct 14, 2024
1 check passed
aphillips added a commit that referenced this pull request Oct 26, 2024
* Create notes-2024-08-19.md

* Accept attributes design & remove spec note (#845)

* Accept attributes design & remove spec note

* Disallow duplicate attribute names (closes #756)

* Add link to contextual options PR

* Add more prose to tag example text

Co-authored-by: Addison Phillips <[email protected]>

* Mention attribute validity condition in the **_valid_** definition

---------

Co-authored-by: Addison Phillips <[email protected]>

* Update selection-declaration design doc based on mtg / issue discussion (#867)

* Add tests for pattern selection (#863)

* Add tests for pattern selection

* Add missing errors

* Apply suggestions from code review

Co-authored-by: Addison Phillips <[email protected]>

---------

Co-authored-by: Addison Phillips <[email protected]>

* Add Duplicate Variant to table in test/README.md (#861)

* Add new selection-declaration alternative: Require annotation of selector variables in placeholders (#860)

* Add new selection-declaration alternative: Require annotation of selector variables in placeholders

* Improve examples

* Switch example order

* Update the stability policy (#834)

* Update the stability policy

Based on discussion in the 2024-07-22 call and in PR #829, update the stability policy.

* A deeper, more thorough rewrite

- Standardizes the phrasing completely.
- Moves all potential future changes (which are not, after all, stability policies) to an "important" block
- Removes duplication
- Separates functions, options, and option values into separate guarantees
- Clarifies the note about formatting changing over time

* Update spec/README.md

Co-authored-by: Tim Chevalier <[email protected]>

* Update spec/README.md

Co-authored-by: Eemeli Aro <[email protected]>

* remove well-formed

* Update spec/README.md

---------

Co-authored-by: Tim Chevalier <[email protected]>
Co-authored-by: Eemeli Aro <[email protected]>

* Refine error handling text (#816)

* Refine error handling text

* Apply suggestions from code review

Co-authored-by: Addison Phillips <[email protected]>

* Update fallback text

* Turn bullet point list into paragraphs

* Be more mighty

Co-authored-by: Addison Phillips <[email protected]>

---------

Co-authored-by: Addison Phillips <[email protected]>

* Create notes-2024-08-26.md

* Select "Match on variables instead of expressions" for selection-declarations (#824)

* Select "Match on variables instead of expressions" for selection-declarations

* Add hybrid option to selection-declaration.md (#870)

* Add hybrid option to selection-declaration.md

* Update selection-declaration.md

fixed glitch in original edit

* Update selection-declaration.md

* Apply suggestions from code review

Fixing typos

Co-authored-by: Addison Phillips <[email protected]>

* Update selection-declaration.md

* Update exploration/selection-declaration.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update exploration/selection-declaration.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update exploration/selection-declaration.md

Co-authored-by: Eemeli Aro <[email protected]>

---------

Co-authored-by: Addison Phillips <[email protected]>
Co-authored-by: Eemeli Aro <[email protected]>

* Update selection-declaration.md

---------

Co-authored-by: Mark Davis <[email protected]>
Co-authored-by: Addison Phillips <[email protected]>

* Fix "Allow immutable input declarative selectors" example (#874)

* Update README.md (#875)

* Update README.md

* Update README.md

* [DESIGN] Update bidi design document to show proposed design (#871)

* [DESIGN] Update bidi design document to show proposed design

The design I actually think we should adopt is the "hybrid approaches" one. This is a necessary first step on the highway to UAX31 compliance and I think is responsibly contained/managed. It is a hybrid approach, in that it permits testable strict implementations to be created (particularly for message serialization).

This PR consists of moving text around. I added one "pro" to one option also.

* Address comments

* Miscellaneous test fixes (#862)

* Add missing expected bad-selector errors

* Fix expected parts for unsupported-statement test

* Add a few new tests for leading-whitespace and duplicate-variant

* Add tests for escaped-char changes made in #743

* Fix tests for attributes with variable values

* Update contributing and joining info (#876)

* Update contributing and joining info

* Update README.md

* Update CONTRIBUTING.md

* Restore CLA copy

* Clarify error & fallback handling (#879)

* Clarify error & fallback handling

* Apply suggestions from code review

Co-authored-by: Addison Phillips <[email protected]>

* Select last rather than first attribute

* Drop mention of "starting with Pattern Selection"

* Attributes can't change the formatted output

* Use "nor" instead of "or" regarding attribute restrictions

---------

Co-authored-by: Addison Phillips <[email protected]>

* Clarify rule selection (#878)

* Clarify rule selection

Fixes #868 

This adds normative SHOULD language to using CLDR plural and ordinal data, which was intended originally.

- clarifies that keyword selection follows exact match
- clarifies the purpose of rule-based selection
- makes non-CLDR-based implementation permitted

* Update spec/registry.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <[email protected]>

---------

Co-authored-by: Eemeli Aro <[email protected]>

* [DESIGN] Maintaining the Standard, Optional and Unicode Namespace Function Sets (#634)

* Design doc to capture registry maintenance

* Update maintaining-registry.md

* Update exploration/maintaining-registry.md

Co-authored-by: Tim Chevalier <[email protected]>

* Update exploration/maintaining-registry.md

Co-authored-by: Tim Chevalier <[email protected]>

* Add user stories, small updates to RGI

* Update exploration/maintaining-registry.md

* Adding additional detail

* Remove machine readable registry; update prose

* Update maintaining-registry.md

* Further development work

* Update to change format and naming

Per the 2024-08-19 call, we decided to switch towards a specification-per-function model, with statuses. This commit includes the initial set of changes to try and implement this.

* Address some comments.

---------

Co-authored-by: Tim Chevalier <[email protected]>

* Create notes-2024-09-09.md

* Fix a typo in an example (#880)

The upcoming work to implement resolved value might make this patch unnecessary or obsolete, but fixing the typo (missing `{`/`}` around the variable in the pattern) just in case

* Remove forward-compatibility promise and all reserved & private syntax (#883)

* Remove forwards compatibility from stability guarantee

* Drop reserved statements and expressions

* Drop private-use annotations

* Update tests

* Clarify that deprecation is not removal

* Match on variables instead of expressions (#877)

* Match on variables instead of expressions

* Apply suggestions from code review

Co-authored-by: Addison Phillips <[email protected]>

* Apply suggestions from code review

* Add missing test changes noticed during implementation

* Empty commit to re-trigger CLA check

---------

Co-authored-by: Addison Phillips <[email protected]>

* Create notes-2024-09-10.md

* Add bidi support and address UAX31/UTS55 requirements (#884)

* Add bidi support and address UAX31/UTS55 requirements

Adds the bidi strong marks ALM, RLM, and LRM plus the bidi isolate controls LRI, RLI, FSI, and PDI to the syntax.

Formally defines optional vs. non-optional whitespace.

Non-optional whitespace must include at least one whitespace character. Optional whitespace may contain only bidi marks (which are invisible)

* Update syntax.md including text from previous PR

* Repair the guidance on strongly directional marks

Include ALM and better specify how to use the marks.

* Fix formatting of the "important"

* Add bidi characters to description of whitespace.

* Permit bidi in a few more places

Add optional whitespace at the start of `variant`

Add optional whitespace around `quoted-pattern`

These changes result in allowing bidi around keys and quoted patterns as intended.

* Update syntax.md ABNF

* Update formatting.md

- Add a note about the difference between formatting and message syntax.
- Clarify the sentence about message directionality.

* Address comment about name/identifier

* Address comments related to bidi in `name`

* Fix variable's location

* Address comment about the list of LRI/PDI targets

* One character typo :-P

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <[email protected]>

* Address comments about rule R3a-1

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <[email protected]>

* Address comment about U+061C

* Change [o]wsp => `o` or `s`

* Match syntax spec to abnf

* Remove *

* Update syntax.md

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/message.abnf

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/message.abnf

Co-authored-by: Eemeli Aro <[email protected]>

* Update syntax.md

* Update spec/message.abnf

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <[email protected]>

---------

Co-authored-by: Eemeli Aro <[email protected]>

* Specify `bad-option` for bad digit size option values (#882)

* Specify `bad-option` for bad digit size option values

Fixes #739

* adopt 'non-negative integer'

* Create notes-2024-09-16.md

* Address name and literal equality (#885)

* Address name and literal equality

This change defines equality as discussed in the 2024-09-09 teleconference in the following ways:

- It defines _name_ equality as being under NFC
- It defines _literal_ equality as explicitly **not** under NFC
- It moves _name_ before _identifier_ in that section of text to avoid a forward definition.

Note that this deviates from discussion in 2024-09-09's call in that we didn't discuss literals at length. It also doesn't discuss non-name/non-literal values, which I'll point out are limited to ASCII sequences such as keywords.

* Typo fix

* Add a note about not requiring implementations to actually normalize

* Implement changes dicussed in 2024-09-16 call.

- Make _key_ require NFC for uniqueness/comparison
- Add a note about NFC
- Make _literal_ **_not_** define equality
- Make text in _name_ identical to that in _key_ for consistency

* Update formatting.md to include keys in NFC

* Address comments

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <[email protected]>

---------

Co-authored-by: Eemeli Aro <[email protected]>

* Update list of normative changes during the LDML45 period (#890)

* Fix typos in data-model-errors tests (#892)

Fix #886

* Update note on exact numeric match for v46 (#891)

Addresses #887 

Non-normative changes to the notes specifically part of LDML46

* Fix attribute value to be literal (#894)

Fixes #893

* Create notes-2024-09-30.md

* Add Resolved Values and Function Handler sections to formatting (#728)

* Add Resolved Values section to formatting

* Apply suggestions from code review

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Tim Chevalier <[email protected]>

* Linkify "resolved value"

* Add some examples & explicitly allow wrapping input values

* No throw, only emit

Co-authored-by: Tim Chevalier <[email protected]>

* Add section on Function Handlers, defining the term

* Apply suggestions from code review

* Rephrase initial resolved value definition

* Update spec/formatting.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update resolved value definition again

Co-authored-by: Addison Phillips <[email protected]>

---------

Co-authored-by: Tim Chevalier <[email protected]>
Co-authored-by: Addison Phillips <[email protected]>

* Define function composition for :number and :integer values (#823)

* Define function composition for :number and :integer values

* Apply suggestions from code review

Co-authored-by: Addison Phillips <[email protected]>

* Add operand option priority example

* Add apostrophes'

Co-authored-by: Tim Chevalier <[email protected]>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <[email protected]>

---------

Co-authored-by: Addison Phillips <[email protected]>
Co-authored-by: Tim Chevalier <[email protected]>

* Create notes-2024-10-07.md

* Apply NFC normalization during :string key comparison (#905)

* Apply NFC normalization during :string key comparison

* Add link to UAX#15

Co-authored-by: Addison Phillips <[email protected]>

---------

Co-authored-by: Addison Phillips <[email protected]>

* Add tests for changes due to bidi/whitespace (#902)

* Add tests for changes due to bidi/whitespace

* Correct output

* Make erroneous test a syntax error

* Define function composition for date/time values (#814)

* Define function composition for date/time values

* Apply suggestions from code review

Co-authored-by: Stanisław Małolepszy <[email protected]>

* Drop the "only"

* Update spec/registry.md

* Update spec/registry.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <[email protected]>

* Make :date and :time composition implementation-defined

---------

Co-authored-by: Stanisław Małolepszy <[email protected]>
Co-authored-by: Addison Phillips <[email protected]>

* DESIGN: Add alternative designs to the design doc on function composition (#806)

* DESIGN: Add a sequel to the design doc on function composition

This document sketches out some alternatives for the machinery
provided to enable function composition.

The goal is to provide an exhaustive list of alternatives.

* Remove 'part 2' document and move contents to the end of part 1

* Revise introduction to reflect the changed goal

* Edited for conciseness

* Further edits for conciseness

* Give a name to InputType and use it

* Refer to motivating examples

* Update function-composition-part-1.md status

Per 2024-10-14 telecon

* Create notes-2024-10-14.md

* Add test for :integer and :number composition (#907)

* Fix `:integer` option `useGrouping` values (#912)

I noticed that `:integer` does not include the "never" value for the option `useGrouping`. This is a bug.

* Drop syntax note on additional bidi changes (#910)

Drop syntax note on addition bidi changes

* Add tests for changes due to #885 (name/literal equality) (#904)

* Add tests for changes due to #885 (name/literal equality)

* Update test/tests/functions/string.json

Co-authored-by: Eemeli Aro <[email protected]>

* Update test/tests/syntax.json

Co-authored-by: Eemeli Aro <[email protected]>

* Update test/tests/functions/string.json

Co-authored-by: Eemeli Aro <[email protected]>

* Added tests for reordering and special case mapping

* Add another selection test

---------

Co-authored-by: Eemeli Aro <[email protected]>

* Add u: options namespace (#846)

* Move spec/registry.md -> spec/registry/default.md

* Add Unicode Registry definition

* Refer to BCP47, add note about only requiring normal tags

* Call it a namespace

* Apply suggestions from code review

Co-authored-by: Addison Phillips <[email protected]>

* Fix test file reference

Co-authored-by: Tim Chevalier <[email protected]>

* Apply suggestions from code review

* Update spec/u-namespace.md

Co-authored-by: Eemeli Aro <[email protected]>

* Apply suggestions from code review

Co-authored-by: Addison Phillips <[email protected]>

* Apply suggestions from code review

Co-authored-by: Addison Phillips <[email protected]>

* Add mention of functions to namespace description

---------

Co-authored-by: Addison Phillips <[email protected]>
Co-authored-by: Tim Chevalier <[email protected]>

* Define function composition for :string values (#798)

* Define function composition for :string values

* Update spec/registry.md as suggested by @stasm in #814

* Drop the "only"

* Update text following code review comments

---------

Co-authored-by: Addison Phillips <[email protected]>

* Drop data model request for feedback on "name" (#909)

* Allow surrogates in content, issue #895 (#906)

* Allow surrogates in content, issue #895

* Grammar and typos, linkify terms, make into a note, and fix 2119 keywords

Thanks Addison!

Co-authored-by: Addison Phillips <[email protected]>

* Not using "localizable elements"

Co-authored-by: Addison Phillips <[email protected]>

* Keep syntax.md in sync with message.abnf

* Added note about surrogates to quoted literals

* Moved the note about surrogates from Security Considerations to The Message

* Update spec/syntax.md

* Update spec/syntax.md

* Italicize  in a couple of places

* Implemeted more (all?) feedback from review

---------

Co-authored-by: Addison Phillips <[email protected]>

---------

Co-authored-by: Eemeli Aro <[email protected]>
Co-authored-by: Elango Cheran <[email protected]>
Co-authored-by: Tim Chevalier <[email protected]>
Co-authored-by: Mark Davis <[email protected]>
Co-authored-by: Danny Gleckler <[email protected]>
Co-authored-by: Steven R. Loomis <[email protected]>
Co-authored-by: Stanisław Małolepszy <[email protected]>
Co-authored-by: Eemeli Aro <[email protected]>
Co-authored-by: Mihai Nita <[email protected]>
aphillips added a commit that referenced this pull request Nov 4, 2024
* [DESIGN] Number selection design refinements

This is to build up and capture technical considerations for how to address the issues raised by @eemeli's PR #842.

* Update examples to match changes to syntax

Also responds to the long discussion with @eemeli about significant digits by removing from the example.

* Address 2024-09-16 call comments

This changes the status to "Re-Opened" and adds a link to the PR. Expect to merge this imminently, although discussion on number selection remains.

* Update exploration/number-selection.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update from main (#914)

* Create notes-2024-08-19.md

* Accept attributes design & remove spec note (#845)

* Accept attributes design & remove spec note

* Disallow duplicate attribute names (closes #756)

* Add link to contextual options PR

* Add more prose to tag example text

Co-authored-by: Addison Phillips <[email protected]>

* Mention attribute validity condition in the **_valid_** definition

---------

Co-authored-by: Addison Phillips <[email protected]>

* Update selection-declaration design doc based on mtg / issue discussion (#867)

* Add tests for pattern selection (#863)

* Add tests for pattern selection

* Add missing errors

* Apply suggestions from code review

Co-authored-by: Addison Phillips <[email protected]>

---------

Co-authored-by: Addison Phillips <[email protected]>

* Add Duplicate Variant to table in test/README.md (#861)

* Add new selection-declaration alternative: Require annotation of selector variables in placeholders (#860)

* Add new selection-declaration alternative: Require annotation of selector variables in placeholders

* Improve examples

* Switch example order

* Update the stability policy (#834)

* Update the stability policy

Based on discussion in the 2024-07-22 call and in PR #829, update the stability policy.

* A deeper, more thorough rewrite

- Standardizes the phrasing completely.
- Moves all potential future changes (which are not, after all, stability policies) to an "important" block
- Removes duplication
- Separates functions, options, and option values into separate guarantees
- Clarifies the note about formatting changing over time

* Update spec/README.md

Co-authored-by: Tim Chevalier <[email protected]>

* Update spec/README.md

Co-authored-by: Eemeli Aro <[email protected]>

* remove well-formed

* Update spec/README.md

---------

Co-authored-by: Tim Chevalier <[email protected]>
Co-authored-by: Eemeli Aro <[email protected]>

* Refine error handling text (#816)

* Refine error handling text

* Apply suggestions from code review

Co-authored-by: Addison Phillips <[email protected]>

* Update fallback text

* Turn bullet point list into paragraphs

* Be more mighty

Co-authored-by: Addison Phillips <[email protected]>

---------

Co-authored-by: Addison Phillips <[email protected]>

* Create notes-2024-08-26.md

* Select "Match on variables instead of expressions" for selection-declarations (#824)

* Select "Match on variables instead of expressions" for selection-declarations

* Add hybrid option to selection-declaration.md (#870)

* Add hybrid option to selection-declaration.md

* Update selection-declaration.md

fixed glitch in original edit

* Update selection-declaration.md

* Apply suggestions from code review

Fixing typos

Co-authored-by: Addison Phillips <[email protected]>

* Update selection-declaration.md

* Update exploration/selection-declaration.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update exploration/selection-declaration.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update exploration/selection-declaration.md

Co-authored-by: Eemeli Aro <[email protected]>

---------

Co-authored-by: Addison Phillips <[email protected]>
Co-authored-by: Eemeli Aro <[email protected]>

* Update selection-declaration.md

---------

Co-authored-by: Mark Davis <[email protected]>
Co-authored-by: Addison Phillips <[email protected]>

* Fix "Allow immutable input declarative selectors" example (#874)

* Update README.md (#875)

* Update README.md

* Update README.md

* [DESIGN] Update bidi design document to show proposed design (#871)

* [DESIGN] Update bidi design document to show proposed design

The design I actually think we should adopt is the "hybrid approaches" one. This is a necessary first step on the highway to UAX31 compliance and I think is responsibly contained/managed. It is a hybrid approach, in that it permits testable strict implementations to be created (particularly for message serialization).

This PR consists of moving text around. I added one "pro" to one option also.

* Address comments

* Miscellaneous test fixes (#862)

* Add missing expected bad-selector errors

* Fix expected parts for unsupported-statement test

* Add a few new tests for leading-whitespace and duplicate-variant

* Add tests for escaped-char changes made in #743

* Fix tests for attributes with variable values

* Update contributing and joining info (#876)

* Update contributing and joining info

* Update README.md

* Update CONTRIBUTING.md

* Restore CLA copy

* Clarify error & fallback handling (#879)

* Clarify error & fallback handling

* Apply suggestions from code review

Co-authored-by: Addison Phillips <[email protected]>

* Select last rather than first attribute

* Drop mention of "starting with Pattern Selection"

* Attributes can't change the formatted output

* Use "nor" instead of "or" regarding attribute restrictions

---------

Co-authored-by: Addison Phillips <[email protected]>

* Clarify rule selection (#878)

* Clarify rule selection

Fixes #868 

This adds normative SHOULD language to using CLDR plural and ordinal data, which was intended originally.

- clarifies that keyword selection follows exact match
- clarifies the purpose of rule-based selection
- makes non-CLDR-based implementation permitted

* Update spec/registry.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <[email protected]>

---------

Co-authored-by: Eemeli Aro <[email protected]>

* [DESIGN] Maintaining the Standard, Optional and Unicode Namespace Function Sets (#634)

* Design doc to capture registry maintenance

* Update maintaining-registry.md

* Update exploration/maintaining-registry.md

Co-authored-by: Tim Chevalier <[email protected]>

* Update exploration/maintaining-registry.md

Co-authored-by: Tim Chevalier <[email protected]>

* Add user stories, small updates to RGI

* Update exploration/maintaining-registry.md

* Adding additional detail

* Remove machine readable registry; update prose

* Update maintaining-registry.md

* Further development work

* Update to change format and naming

Per the 2024-08-19 call, we decided to switch towards a specification-per-function model, with statuses. This commit includes the initial set of changes to try and implement this.

* Address some comments.

---------

Co-authored-by: Tim Chevalier <[email protected]>

* Create notes-2024-09-09.md

* Fix a typo in an example (#880)

The upcoming work to implement resolved value might make this patch unnecessary or obsolete, but fixing the typo (missing `{`/`}` around the variable in the pattern) just in case

* Remove forward-compatibility promise and all reserved & private syntax (#883)

* Remove forwards compatibility from stability guarantee

* Drop reserved statements and expressions

* Drop private-use annotations

* Update tests

* Clarify that deprecation is not removal

* Match on variables instead of expressions (#877)

* Match on variables instead of expressions

* Apply suggestions from code review

Co-authored-by: Addison Phillips <[email protected]>

* Apply suggestions from code review

* Add missing test changes noticed during implementation

* Empty commit to re-trigger CLA check

---------

Co-authored-by: Addison Phillips <[email protected]>

* Create notes-2024-09-10.md

* Add bidi support and address UAX31/UTS55 requirements (#884)

* Add bidi support and address UAX31/UTS55 requirements

Adds the bidi strong marks ALM, RLM, and LRM plus the bidi isolate controls LRI, RLI, FSI, and PDI to the syntax.

Formally defines optional vs. non-optional whitespace.

Non-optional whitespace must include at least one whitespace character. Optional whitespace may contain only bidi marks (which are invisible)

* Update syntax.md including text from previous PR

* Repair the guidance on strongly directional marks

Include ALM and better specify how to use the marks.

* Fix formatting of the "important"

* Add bidi characters to description of whitespace.

* Permit bidi in a few more places

Add optional whitespace at the start of `variant`

Add optional whitespace around `quoted-pattern`

These changes result in allowing bidi around keys and quoted patterns as intended.

* Update syntax.md ABNF

* Update formatting.md

- Add a note about the difference between formatting and message syntax.
- Clarify the sentence about message directionality.

* Address comment about name/identifier

* Address comments related to bidi in `name`

* Fix variable's location

* Address comment about the list of LRI/PDI targets

* One character typo :-P

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <[email protected]>

* Address comments about rule R3a-1

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <[email protected]>

* Address comment about U+061C

* Change [o]wsp => `o` or `s`

* Match syntax spec to abnf

* Remove *

* Update syntax.md

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/message.abnf

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/message.abnf

Co-authored-by: Eemeli Aro <[email protected]>

* Update syntax.md

* Update spec/message.abnf

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <[email protected]>

---------

Co-authored-by: Eemeli Aro <[email protected]>

* Specify `bad-option` for bad digit size option values (#882)

* Specify `bad-option` for bad digit size option values

Fixes #739

* adopt 'non-negative integer'

* Create notes-2024-09-16.md

* Address name and literal equality (#885)

* Address name and literal equality

This change defines equality as discussed in the 2024-09-09 teleconference in the following ways:

- It defines _name_ equality as being under NFC
- It defines _literal_ equality as explicitly **not** under NFC
- It moves _name_ before _identifier_ in that section of text to avoid a forward definition.

Note that this deviates from discussion in 2024-09-09's call in that we didn't discuss literals at length. It also doesn't discuss non-name/non-literal values, which I'll point out are limited to ASCII sequences such as keywords.

* Typo fix

* Add a note about not requiring implementations to actually normalize

* Implement changes dicussed in 2024-09-16 call.

- Make _key_ require NFC for uniqueness/comparison
- Add a note about NFC
- Make _literal_ **_not_** define equality
- Make text in _name_ identical to that in _key_ for consistency

* Update formatting.md to include keys in NFC

* Address comments

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/syntax.md

Co-authored-by: Eemeli Aro <[email protected]>

---------

Co-authored-by: Eemeli Aro <[email protected]>

* Update list of normative changes during the LDML45 period (#890)

* Fix typos in data-model-errors tests (#892)

Fix #886

* Update note on exact numeric match for v46 (#891)

Addresses #887 

Non-normative changes to the notes specifically part of LDML46

* Fix attribute value to be literal (#894)

Fixes #893

* Create notes-2024-09-30.md

* Add Resolved Values and Function Handler sections to formatting (#728)

* Add Resolved Values section to formatting

* Apply suggestions from code review

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Tim Chevalier <[email protected]>

* Linkify "resolved value"

* Add some examples & explicitly allow wrapping input values

* No throw, only emit

Co-authored-by: Tim Chevalier <[email protected]>

* Add section on Function Handlers, defining the term

* Apply suggestions from code review

* Rephrase initial resolved value definition

* Update spec/formatting.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update resolved value definition again

Co-authored-by: Addison Phillips <[email protected]>

---------

Co-authored-by: Tim Chevalier <[email protected]>
Co-authored-by: Addison Phillips <[email protected]>

* Define function composition for :number and :integer values (#823)

* Define function composition for :number and :integer values

* Apply suggestions from code review

Co-authored-by: Addison Phillips <[email protected]>

* Add operand option priority example

* Add apostrophes'

Co-authored-by: Tim Chevalier <[email protected]>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <[email protected]>

---------

Co-authored-by: Addison Phillips <[email protected]>
Co-authored-by: Tim Chevalier <[email protected]>

* Create notes-2024-10-07.md

* Apply NFC normalization during :string key comparison (#905)

* Apply NFC normalization during :string key comparison

* Add link to UAX#15

Co-authored-by: Addison Phillips <[email protected]>

---------

Co-authored-by: Addison Phillips <[email protected]>

* Add tests for changes due to bidi/whitespace (#902)

* Add tests for changes due to bidi/whitespace

* Correct output

* Make erroneous test a syntax error

* Define function composition for date/time values (#814)

* Define function composition for date/time values

* Apply suggestions from code review

Co-authored-by: Stanisław Małolepszy <[email protected]>

* Drop the "only"

* Update spec/registry.md

* Update spec/registry.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <[email protected]>

* Update spec/registry.md

Co-authored-by: Eemeli Aro <[email protected]>

* Make :date and :time composition implementation-defined

---------

Co-authored-by: Stanisław Małolepszy <[email protected]>
Co-authored-by: Addison Phillips <[email protected]>

* DESIGN: Add alternative designs to the design doc on function composition (#806)

* DESIGN: Add a sequel to the design doc on function composition

This document sketches out some alternatives for the machinery
provided to enable function composition.

The goal is to provide an exhaustive list of alternatives.

* Remove 'part 2' document and move contents to the end of part 1

* Revise introduction to reflect the changed goal

* Edited for conciseness

* Further edits for conciseness

* Give a name to InputType and use it

* Refer to motivating examples

* Update function-composition-part-1.md status

Per 2024-10-14 telecon

* Create notes-2024-10-14.md

* Add test for :integer and :number composition (#907)

* Fix `:integer` option `useGrouping` values (#912)

I noticed that `:integer` does not include the "never" value for the option `useGrouping`. This is a bug.

* Drop syntax note on additional bidi changes (#910)

Drop syntax note on addition bidi changes

* Add tests for changes due to #885 (name/literal equality) (#904)

* Add tests for changes due to #885 (name/literal equality)

* Update test/tests/functions/string.json

Co-authored-by: Eemeli Aro <[email protected]>

* Update test/tests/syntax.json

Co-authored-by: Eemeli Aro <[email protected]>

* Update test/tests/functions/string.json

Co-authored-by: Eemeli Aro <[email protected]>

* Added tests for reordering and special case mapping

* Add another selection test

---------

Co-authored-by: Eemeli Aro <[email protected]>

* Add u: options namespace (#846)

* Move spec/registry.md -> spec/registry/default.md

* Add Unicode Registry definition

* Refer to BCP47, add note about only requiring normal tags

* Call it a namespace

* Apply suggestions from code review

Co-authored-by: Addison Phillips <[email protected]>

* Fix test file reference

Co-authored-by: Tim Chevalier <[email protected]>

* Apply suggestions from code review

* Update spec/u-namespace.md

Co-authored-by: Eemeli Aro <[email protected]>

* Apply suggestions from code review

Co-authored-by: Addison Phillips <[email protected]>

* Apply suggestions from code review

Co-authored-by: Addison Phillips <[email protected]>

* Add mention of functions to namespace description

---------

Co-authored-by: Addison Phillips <[email protected]>
Co-authored-by: Tim Chevalier <[email protected]>

* Define function composition for :string values (#798)

* Define function composition for :string values

* Update spec/registry.md as suggested by @stasm in #814

* Drop the "only"

* Update text following code review comments

---------

Co-authored-by: Addison Phillips <[email protected]>

* Drop data model request for feedback on "name" (#909)

* Allow surrogates in content, issue #895 (#906)

* Allow surrogates in content, issue #895

* Grammar and typos, linkify terms, make into a note, and fix 2119 keywords

Thanks Addison!

Co-authored-by: Addison Phillips <[email protected]>

* Not using "localizable elements"

Co-authored-by: Addison Phillips <[email protected]>

* Keep syntax.md in sync with message.abnf

* Added note about surrogates to quoted literals

* Moved the note about surrogates from Security Considerations to The Message

* Update spec/syntax.md

* Update spec/syntax.md

* Italicize  in a couple of places

* Implemeted more (all?) feedback from review

---------

Co-authored-by: Addison Phillips <[email protected]>

---------

Co-authored-by: Eemeli Aro <[email protected]>
Co-authored-by: Elango Cheran <[email protected]>
Co-authored-by: Tim Chevalier <[email protected]>
Co-authored-by: Mark Davis <[email protected]>
Co-authored-by: Danny Gleckler <[email protected]>
Co-authored-by: Steven R. Loomis <[email protected]>
Co-authored-by: Stanisław Małolepszy <[email protected]>
Co-authored-by: Eemeli Aro <[email protected]>
Co-authored-by: Mihai Nita <[email protected]>

* Add serialization proposal

* Revert "Add serialization proposal"

This reverts commit 17af553.

* Revert "Update from main (#914)"

This reverts commit da9377b.

* Add serialization proposal

---------

Co-authored-by: Eemeli Aro <[email protected]>
Co-authored-by: Elango Cheran <[email protected]>
Co-authored-by: Tim Chevalier <[email protected]>
Co-authored-by: Mark Davis <[email protected]>
Co-authored-by: Danny Gleckler <[email protected]>
Co-authored-by: Steven R. Loomis <[email protected]>
Co-authored-by: Stanisław Małolepszy <[email protected]>
Co-authored-by: Eemeli Aro <[email protected]>
Co-authored-by: Mihai Nita <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Agenda+ Requested for upcoming teleconference design Design principles, decisions formatting LDML46.1 MF2.0 Draft Candidate normative resolve-candidate This issue appears to have been answered or resolved, and may be closed soon.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants