diff --git a/docs/autocomplete_compatibility.md b/docs/autocomplete_compatibility.md new file mode 100644 index 00000000..9000d4d5 --- /dev/null +++ b/docs/autocomplete_compatibility.md @@ -0,0 +1,41 @@ +# Autocomplete Compatability + +Many lists are used to drive autocompleting user interface components. In order to support this, a standardised concept of an "autocomplete compatible" list has been defined. A list which is autocomplete compatible has the information required to drive an autocomplete-based selection mechanism, and has that information in well-known fields so no additional configuration is required to use that list in an autocomplete mechanism that understands this standard. + +# Specification + +An autocomplete-compatible list must have: + +1. A `name` field, which is a string value in every record. This should be the "best" name for the identified object, which should usually be the one that regular users are most likely to use - this may not always be the official or legal name. For instance, an organisation may have a well-known trading name that is different from their name as registered with Companies House. +2. (Optionally) an `match_synonyms` field which, in every record in which it is present, is an array of strings. These should be alternative names by which the object may be known, but which can only refer to this object. +2. (Optionally) a `suggestion_synonyms` field which, in every record in which it is present, is an array of strings. These should be alternative names by which the object may possibly be known, but which might not always refer to this object. +2. (Optionally) an `hint` field which, in every record in which it is present, is a string. It should be a message to be shown to users to help them decide if this is the object they mean to select. + +In addition, for any given list, the combination of `name` and values inside `match_synonyms` must not contain duplicates - there can be no string that matches more than one record in the `name` field or a `match_synonym`. Duplication within `suggestion_synonyms` is fine. + +## Matching + +A user-inputted string is compared to names from `name`, `match_synonyms` or `suggestion_synonyms` fields to see if they "match" any records. The nature of this comparison is deliberately weakly specified, as autocomplete logic may use arbitrarily clever fuzzy matches. However, it will always include case-insensitive string matching, so there is no need to list multiple names in a record that differ only in case. + +See https://bat-design-history.netlify.app/register-trainee-teachers/autocomplete-improvements/ for a discussion of the development of fuzzy matching in autocompletion engines. + +# Documentation + +A reference data list may declare that it is autocomplete compatible with the following wording: + +``` +This list is [autocomplete compatible](autocomplete_compatability.md). +``` + +# Validation + +The RSpec tests for a list can confirm it's autocomplete compatible with the following pattern: + +```ruby + describe 'qualifications' do + it 'is a valid autocomplete-capable list' do + DfE::ReferenceData::Qualifications::QUALIFICATIONS.validate_autocomplete_compatibility! + end + end + +``` diff --git a/docs/lists_degrees.md b/docs/lists_degrees.md index 1a5991c2..d2c96162 100644 --- a/docs/lists_degrees.md +++ b/docs/lists_degrees.md @@ -18,6 +18,8 @@ Source: https://github.com/DFE-Digital/apply-for-teacher-training-prototype/blob Quality: Manually updated on an ad-hoc basis. Please submit a pull request if inaccuracies or omissions are found. +This list is [autocomplete compatible](autocomplete_compatability.md). + | Field | Type | Purpose | |---|---|---| | `id` | UUID | A unique identifier. The same as `dttp_id` if that field is non-`nil`, otherwise a new UUID was minted at import time. | @@ -49,6 +51,8 @@ Source: https://github.com/DFE-Digital/apply-for-teacher-training-prototype/blob Quality: Manually updated on an ad-hoc basis. Please submit a pull request if inaccuracies or omissions are found. +This list is [autocomplete compatible](autocomplete_compatability.md). + | Field | Type | Purpose | |---|---|---| | `id` | UUID | A unique identifier. The same as `dttp_id` if that field is non-`nil`, otherwise a new UUID was minted at import time. | @@ -78,6 +82,8 @@ Source: Automatically derived from joining the `TYPES` and `GENERIC_TYPES` lists Quality: Automatically derived from the source data, so only as correct as they are. +This list is [autocomplete compatible](autocomplete_compatability.md). + | Field | Type | Purpose | |---|---|---| | `id` | UUID | A unique identifier. The same as `dttp_id` if that field is non-`nil`, otherwise a new UUID was minted at import time. | @@ -101,6 +107,8 @@ Users: Apply team. Quality: Manually updated on an ad-hoc basis. Please submit a pull request if inaccuracies or omissions are found. +This list is [autocomplete compatible](autocomplete_compatability.md). + | Field | Type | Purpose | |---|---|---| | `id` | UUID | A unique identifier. | @@ -126,6 +134,8 @@ Source: https://github.com/DFE-Digital/apply-for-teacher-training-prototype/blob Quality: Manually updated on an ad-hoc basis. Please submit a pull request if inaccuracies or omissions are found. +This list is [autocomplete compatible](autocomplete_compatability.md). + | Field | Type | Purpose | |---|---|---| | `id` | UUID | A unique identifier. The same as `dttp_id` if that field is non-`nil`, otherwise a new UUID was minted at import time. | @@ -151,6 +161,8 @@ Source: https://github.com/DFE-Digital/apply-for-teacher-training-prototype/blob Quality: Manually updated on an ad-hoc basis. Please submit a pull request if inaccuracies or omissions are found. +This list is [autocomplete compatible](autocomplete_compatability.md). + | Field | Type | Purpose | |---|---|---| | `id` | UUID | A unique identifier. The same as `dttp_id` if that field is non-`nil`, otherwise a new UUID was minted at import time. | @@ -175,6 +187,8 @@ Source: Automatically derived from joining the `SINGLE_SUBJECTS` and `COMBINED_S Quality: Automatically derived from the source data, so only as correct as they are. +This list is [autocomplete compatible](autocomplete_compatability.md). + | Field | Type | Purpose | |---|---|---| | `id` | UUID | A unique identifier. The same as `dttp_id` if that field is non-`nil`, otherwise a new UUID was minted at import time. | @@ -201,6 +215,8 @@ Source: https://github.com/DFE-Digital/apply-for-teacher-training-prototype/blob Quality: Manually updated on an ad-hoc basis. Please submit a pull request if inaccuracies or omissions are found. +This list is [autocomplete compatible](autocomplete_compatability.md). + | Field | Type | Purpose | |---|---|---| | `id` | UUID | A unique identifier. The same as `dttp_id` if that field is non-`nil`, otherwise a new UUID was minted at import time. | diff --git a/docs/lists_qualifications.md b/docs/lists_qualifications.md index 6563154c..d5afe35b 100644 --- a/docs/lists_qualifications.md +++ b/docs/lists_qualifications.md @@ -18,6 +18,8 @@ Source: https://www.gov.uk/what-different-qualification-levels-mean/list-of-qual Quality: Manually updated on an ad-hoc basis. Please submit a pull request if inaccuracies or omissions are found. +This list is [autocomplete compatible](autocomplete_compatability.md). + | Field | Type | Purpose | |---|---|---| | `id` | UUID | A unique identifier | diff --git a/lib/dfe/reference_data/degrees/institutions.rb b/lib/dfe/reference_data/degrees/institutions.rb index d8c02597..3367830f 100644 --- a/lib/dfe/reference_data/degrees/institutions.rb +++ b/lib/dfe/reference_data/degrees/institutions.rb @@ -71,8 +71,7 @@ module Degrees suggestion_synonyms: [], match_synonyms: ['University of St Mark and St John', - 'University of Saint Mark and Saint John', - 'University of St Mark and St John'], + 'University of Saint Mark and Saint John'], hesa_itt_code: '14', dttp_id: '3a71f34a-2887-e711-80d8-005056ac45bb', ukprn: '10037449' }, diff --git a/spec/lib/dfe/reference_data/degrees/autocomplete_spec.rb b/spec/lib/dfe/reference_data/degrees/autocomplete_spec.rb new file mode 100644 index 00000000..07017a0e --- /dev/null +++ b/spec/lib/dfe/reference_data/degrees/autocomplete_spec.rb @@ -0,0 +1,5 @@ +RSpec.describe DfE::ReferenceData::Degrees::SINGLE_SUBJECTS do + describe 'single subjects' do + let(:single_subjects) { DfE::ReferenceData::Degrees::SINGLE_SUBJECTS.all } + end +end diff --git a/spec/lib/dfe/reference_data/degrees/combined_subjects_spec.rb b/spec/lib/dfe/reference_data/degrees/combined_subjects_spec.rb index 3c32c35c..0591b71d 100644 --- a/spec/lib/dfe/reference_data/degrees/combined_subjects_spec.rb +++ b/spec/lib/dfe/reference_data/degrees/combined_subjects_spec.rb @@ -1,3 +1,5 @@ +require 'support/autocomplete' + RSpec.describe DfE::ReferenceData::Degrees::COMBINED_SUBJECTS do describe 'subject IDs are correct' do let(:records) { described_class.all } @@ -35,4 +37,6 @@ end end end + + it_should_behave_like 'a valid autocomplete-capable list' end diff --git a/spec/lib/dfe/reference_data/degrees/generic_types.rb b/spec/lib/dfe/reference_data/degrees/generic_types.rb new file mode 100644 index 00000000..a571cbf3 --- /dev/null +++ b/spec/lib/dfe/reference_data/degrees/generic_types.rb @@ -0,0 +1,7 @@ +RSpec.describe DfE::ReferenceData::Degrees::GENERIC_TYPES do + describe 'generic types' do + it 'is a valid autocomplete-capable list' do + DfE::ReferenceData::GENERIC_TYPES.validate_autocomplete_compatibility! + end + end +end diff --git a/spec/lib/dfe/reference_data/degrees/grades_spec.rb b/spec/lib/dfe/reference_data/degrees/grades_spec.rb new file mode 100644 index 00000000..7456bfb6 --- /dev/null +++ b/spec/lib/dfe/reference_data/degrees/grades_spec.rb @@ -0,0 +1,5 @@ +require 'support/autocomplete' + +RSpec.describe DfE::ReferenceData::Degrees::GRADES do + it_should_behave_like 'a valid autocomplete-capable list' +end diff --git a/spec/lib/dfe/reference_data/degrees/institutions_spec.rb b/spec/lib/dfe/reference_data/degrees/institutions_spec.rb new file mode 100644 index 00000000..717fc407 --- /dev/null +++ b/spec/lib/dfe/reference_data/degrees/institutions_spec.rb @@ -0,0 +1,5 @@ +require 'support/autocomplete' + +RSpec.describe DfE::ReferenceData::Degrees::INSTITUTIONS do + it_should_behave_like 'a valid autocomplete-capable list' +end diff --git a/spec/lib/dfe/reference_data/degrees/single_subjects_spec.rb b/spec/lib/dfe/reference_data/degrees/single_subjects_spec.rb index 327767b7..2620399c 100644 --- a/spec/lib/dfe/reference_data/degrees/single_subjects_spec.rb +++ b/spec/lib/dfe/reference_data/degrees/single_subjects_spec.rb @@ -1,3 +1,5 @@ +require 'support/autocomplete' + RSpec.describe DfE::ReferenceData::Degrees::SINGLE_SUBJECTS do describe 'single subjects' do let(:single_subjects) { DfE::ReferenceData::Degrees::SINGLE_SUBJECTS.all } @@ -22,4 +24,6 @@ end end end + + it_should_behave_like 'a valid autocomplete-capable list' end diff --git a/spec/lib/dfe/reference_data/degrees/subjects_spec.rb b/spec/lib/dfe/reference_data/degrees/subjects_spec.rb new file mode 100644 index 00000000..b1da1536 --- /dev/null +++ b/spec/lib/dfe/reference_data/degrees/subjects_spec.rb @@ -0,0 +1,5 @@ +require 'support/autocomplete' + +RSpec.describe DfE::ReferenceData::Degrees::SUBJECTS do + it_should_behave_like 'a valid autocomplete-capable list' +end diff --git a/spec/lib/dfe/reference_data/degrees/types.rb b/spec/lib/dfe/reference_data/degrees/types.rb new file mode 100644 index 00000000..3b470f26 --- /dev/null +++ b/spec/lib/dfe/reference_data/degrees/types.rb @@ -0,0 +1,7 @@ +RSpec.describe DfE::ReferenceData::Degrees::TYPES do + describe 'types' do + it 'is a valid autocomplete-capable list' do + DfE::ReferenceData::TYPES.validate_autocomplete_compatibility! + end + end +end diff --git a/spec/lib/dfe/reference_data/degrees/types_including_generics.rb b/spec/lib/dfe/reference_data/degrees/types_including_generics.rb new file mode 100644 index 00000000..0350b746 --- /dev/null +++ b/spec/lib/dfe/reference_data/degrees/types_including_generics.rb @@ -0,0 +1,7 @@ +RSpec.describe DfE::ReferenceData::Degrees::TYPES_INCLUDING_GENERICS do + describe 'types including generics' do + it 'is a valid autocomplete-capable list' do + DfE::ReferenceData::TYPES_INCLUDING_GENERICS.validate_autocomplete_compatibility! + end + end +end diff --git a/spec/lib/dfe/reference_data/qualifications_spec.rb b/spec/lib/dfe/reference_data/qualifications_spec.rb new file mode 100644 index 00000000..93e7d652 --- /dev/null +++ b/spec/lib/dfe/reference_data/qualifications_spec.rb @@ -0,0 +1,5 @@ +require 'support/autocomplete' + +RSpec.describe DfE::ReferenceData::Qualifications::QUALIFICATIONS do + it_should_behave_like 'a valid autocomplete-capable list' +end diff --git a/spec/support/autocomplete.rb b/spec/support/autocomplete.rb new file mode 100644 index 00000000..6a33f207 --- /dev/null +++ b/spec/support/autocomplete.rb @@ -0,0 +1,51 @@ +# Seriously, rubocop, this is not a complicated method +# rubocop:disable Metrics/AbcSize +# rubocop:disable Metrics/CyclomaticComplexity +# rubocop:disable Metrics/PerceivedComplexity + +def validate_autocomplete_compatible_record!(record) + data = record.data + + # Basic validation: has name, match_synonyms, suggestion_synonyms, hint of appropriate types + # TODO: Rewrite this into a one-off schema check when schemas are merged in + throw "Record #{record.id} lacks a name string" unless !data[:name].nil? && data[:name].is_a?(String) + throw "Record #{record.id} has a non-string hint" unless data[:hint].nil? || data[:hint].is_a?(String) + ms = data[:match_synonyms] + throw "Record #{record.id} has non-array-of-strings match synonyms" unless ms.nil? || ms.is_a?(Array) || ms.all? { |e| e.is_a?(String) } + ss = data[:suggestion_synonyms] + throw "Record #{record.id} has non-array-of-strings suggestion synonyms" unless ss.nil? || ss.is_a?(Array) || ss.all? { |e| e.is_a?(String) } +end + +# rubocop:enable Metrics/AbcSize +# rubocop:enable Metrics/CyclomaticComplexity +# rubocop:enable Metrics/PerceivedComplexity + +RSpec::Matchers.define :be_valid_autocomplete_compatible do + match do |actual| + # Keep track of names and match synonyms that have been used in previous + # records, to ensure uniqueness + used_names = {} + + actual.all.each do |record| + validate_autocomplete_compatible_record!(record) + + # Unique name check + names = Array.new(record.match_synonyms || []) + names << record.name + + names.each do |name| + throw "Record #{record.id} has name/match synonym #{name} which is already used by record #{used_names[name]}" if used_names[name] + used_names[name] = record.id + end + end + + # Ok, I guess we're good then! + true + end +end + +RSpec.shared_examples 'a valid autocomplete-capable list' do + it 'is a valid autocomplete-capable list' do + expect(described_class).to be_valid_autocomplete_compatible + end +end