Switch fully to documents and editions #683

thomasleese · 2016-12-20T14:24:50Z

Second part of the documents and editions experience. Here we remove the locale and content_id columns from content items and switch to using only documents as the canonical source of data.

There are no migrations so this should be easy to deploy after #705.

kevindew · 2016-12-20T15:23:37Z

Looks good Tom - think you might want to separate that migration out though as the update and inserts will be slow queries. Probably want to populate this slowly in the same way we've done the pillars.

binaryberry

Paired on reviewing this with @boffbowsh , as a mentoring exercise. I'm off for Xmas now but he says he's happy to answer any questions!

binaryberry · 2016-12-22T14:23:35Z

db/schema.rb

@@ -50,7 +50,6 @@
  end

  create_table "content_items", force: :cascade do |t|
-    t.string   "content_id"


The removal of these lines in the schema seems to be the consequence of another migration - could you check the source of these lines?

binaryberry · 2016-12-22T14:25:33Z

db/migrate/20161220134413_create_documents.rb

+  end
+
+  def down
+    remove_index :documents, [:content_id, :locale]


Removing the index isn't necessary if you're dropping the documents table.
Also, instead of up and down, you could have a single change method.

binaryberry · 2016-12-22T14:35:04Z

db/migrate/20161221074658_create_editions.rb

+             )'
+
+    remove_column :content_items, :content_id
+    remove_column :content_items, :locale


The code that uses these columns needs to be changed to get the content_id and locale from documents before we can remove the columns.

binaryberry · 2016-12-22T14:43:41Z

db/migrate/20161220134413_create_documents.rb

+  def up
+    create_table :documents do |t|
+      t.string :content_id
+      t.string :locale


Both of these should be not null.

binaryberry · 2016-12-22T14:50:16Z

app/presenters/queries/content_item_presenter.rb

@@ -102,7 +102,9 @@ def select_fields(scope)
          when :base_path
            "content_items.base_path as base_path"
          when :locale
-            "content_items.locale as locale"
+            "documents_content_items.locale as locale"


where is documents_content_items coming from?

This was the name that Rails gave to the join of documents in this case, because there was a subquery with a join on the name documents already. I agree it's unclear though and I don't really like it, will try and find a way to make it clearer.

binaryberry · 2016-12-22T14:52:47Z

app/queries/get_content.rb

@@ -3,7 +3,8 @@ module GetContent
    def self.call(content_id, locale = nil, version: nil, include_warnings: false)
      locale_to_use = locale || ContentItem::DEFAULT_LOCALE

-      content_items = ContentItem.where(content_id: content_id, locale: locale_to_use)
+      content_items = ContentItem.joins(:document).
+        where('documents.content_id': content_id, 'documents.locale': locale_to_use)


You can also use:

where(documents: {content_id: content_id, locale: locale_to_use})

binaryberry · 2016-12-22T14:53:16Z

app/queries/get_content_collection.rb

@@ -59,7 +59,7 @@ def content_items
      scope = ContentItem.where(document_type: lookup_document_types)
      scope = scope.where(publishing_app: publishing_app) if publishing_app
      scope = scope.where(state: states) if states.present?
-      scope = scope.where(locale: locale) unless locale == "all"
+      scope = scope.joins(:document).where('documents.locale': locale) unless locale == "all"


Similar to above

kevindew · 2017-01-03T11:21:10Z

app/commands/v2/discard_draft.rb

@@ -62,28 +62,24 @@ def increment_live_lock_version
        lock_version.save!
      end

+      def document
+        @document ||= Queries::GetDocument.(content_id, locale)


Can access these values straight from payload so hopefully we can kill of the need for individual methods for them?

kevindew · 2017-01-03T11:21:59Z

app/commands/v2/discard_draft.rb

+      def document
+        @document ||= Queries::GetDocument.(content_id, locale)
+      end
+
      def draft


Can we kill this method and just use document.draft?

kevindew · 2017-01-03T11:22:25Z

app/commands/v2/discard_draft.rb

-          locale: locale,
-          state: "draft",
-        )
+        @draft ||= document.draft
      end

      def live


Can we kill this method and just use document.live?

kevindew · 2017-01-03T11:26:05Z

app/commands/v2/put_content.rb

-      def pessimistic_content_item_scope
-        ContentItem.where(content_id: content_id, locale: locale).lock
+      def document
+        @document ||= Queries::GetDocument.(content_id, locale)
      end

      def previously_published_item
        @previously_published_item ||=


can we use document.live here?

kevindew · 2017-01-03T11:27:16Z

app/commands/v2/put_content.rb

            state: "draft",
            content_store: "draft",
            user_facing_version: content_item.user_facing_version,
          )
        )
+
+        # update these fields to also update the underlying document


Does this need marking as temporary code?

kevindew · 2017-01-03T11:33:03Z

app/models/content_item.rb

@@ -96,6 +94,29 @@ class ContentItem < ApplicationRecord
    end
  end

+  def as_json(options = {})
+    super(options).merge(content_id: content_id, locale: locale)


Should these lookups be prefixed with document

kevindew · 2017-01-03T11:34:12Z

app/models/content_item.rb

@@ -112,16 +133,14 @@ def draft_cannot_be_behind_live
    if state == "draft"
      draft_version = user_facing_version


Can we simplify this to be document.live.user_facing_version?

kevindew · 2017-01-03T11:37:42Z

app/models/document.rb

+    message: 'must be a supported locale'
+  }
+
+  def draft


I think we can probably improve this somehow, maybe using a scope or similar. Do you think we should memoize here? I guess we hit it a lot, probably should act like a relation (I've been considering it one in my dozy just back from New Year state heh)

kevindew · 2017-01-03T11:40:24Z

app/queries/get_document.rb

@@ -0,0 +1,25 @@
+module Queries
+  module GetDocument


I think this should be named to indicate the lock like Queries::LockedDocument?

kevindew · 2017-01-03T11:41:15Z

app/queries/get_document.rb

+          Document.find_or_create_by!(
+            content_id: content_id,
+            locale: locale_to_use
+          ).lock!


I reckon we can refactor this to not have a second query for the lock. Will have a think.

Now called import_content_id_events which is more descriptive of what the task actually does.

This data will have been "cleaned up" many moons ago by now and this task is no longer needed.

As part of the split of ContentItems into Documents and Editions.

These monkey patches are used so that we can continue using a polymorphic association on LockVersion to ContentItem/Edition without changing what is saved in the database currently - namely a `target_type` of "ContentItem". This allows us to swap from this branch to the currently released version without pain. This patching should live for a very short period of time as we will not need to maintain backwards compatibility once this is deployed and we also intend to drop the lock_version table.

These are changes further to the initial rebasing of #729 which make better use of document.

Since we can and do validate UUID's with a regex it makes sense for us to use the correct format for these as a route constraint.

Changed to no longer refer to content_item, also made more succinct.

These clear up the cases where the factory is on the next line so they are all consistently on the same line as the method opening.

Although valid syntax in Ruby this is inconsistent with our usage of methods.

As ActiveRecord handles the memoization for us it seems redundant to have two levels of this in.

This code would have accidentally been concealing errors.

This replaces the many places where we run `joins(:document)`.

A small refactoring to try make a particularly difficult line of code to follow easier to understand.

I think this is easier to follow than the concatentation approach it replaces, still not super easy but seemed less heavy duty than squigly heredoc.

This makes the matcher more specific. It also removes the strings that are output on a failure. We don't use them in other places and in this instance they don't seem particularly useful.

For usage with Documents and Editions.

This is no longer used.

kevindew · 2017-01-26T10:30:11Z

Thanks @dougdroper - Just going to run this on integration for an hour before merging. I'll see if I can get a deploy slot for today.

This method was renamed in the past (#683), but the relevant documentation still has the old method name.

This method was renamed in the past (#683), but the relevant documentation still has the old method name. Co-authored-by: Cristina <[email protected]>

thomasleese force-pushed the documents-and-editions branch from 7c5d58f to 8e3366d Compare December 20, 2016 14:39

thomasleese force-pushed the documents-and-editions branch 2 times, most recently from 8556833 to 2207ad6 Compare December 21, 2016 08:19

kevindew force-pushed the db-pillars-integration branch from c2bbf82 to c0b21f5 Compare December 21, 2016 10:57

thomasleese force-pushed the documents-and-editions branch 3 times, most recently from ec1dfce to eeafeec Compare December 21, 2016 11:56

thomasleese changed the base branch from db-pillars-integration to master December 21, 2016 11:57

thomasleese force-pushed the documents-and-editions branch from 43f7c3e to 0e88327 Compare December 22, 2016 09:58

binaryberry reviewed Dec 22, 2016

View reviewed changes

thomasleese force-pushed the documents-and-editions branch 4 times, most recently from 8f180a9 to 76134ea Compare December 23, 2016 11:06

thomasleese force-pushed the documents-and-editions branch from c1a117a to 211648c Compare December 30, 2016 09:57

kevindew reviewed Jan 3, 2017

View reviewed changes

thomasleese force-pushed the documents-and-editions branch 6 times, most recently from c379d3c to 84e1b5f Compare January 4, 2017 14:39

thomasleese changed the title ~~[WIP] Documents and Editions~~ Documents and Editions Jan 4, 2017

thomasleese force-pushed the documents-and-editions branch 6 times, most recently from 9c65e73 to 7dc077e Compare January 6, 2017 10:58

kevindew added 20 commits January 26, 2017 10:20

rename import_content_item_events task

48ed13e

Now called import_content_id_events which is more descriptive of what the task actually does.

Use editions rather than content_items in govspeak compare task

eac1ab1

Use editions rather than content items in descriptions

c3f55ec

Remove data_cleanup task for Specialist Publisher

0113dcd

This data will have been "cleaned up" many moons ago by now and this task is no longer needed.

Change content_item variable names to edition

8601584

As part of the split of ContentItems into Documents and Editions.

Use document in put_content refactoring

8fb56ac

These are changes further to the initial rebasing of #729 which make better use of document.

Validate UUID at route level

8f3491e

Since we can and do validate UUID's with a regex it makes sense for us to use the correct format for these as a route constraint.

Update CheckForContentItemPreventingDraftFromBeingPublished class name

cb03463

Changed to no longer refer to content_item, also made more succinct.

Make FactoryGirl initialisation format consistent

77265ed

These clear up the cases where the factory is on the next line so they are all consistently on the same line as the method opening.

Add missing brackets to method calls

8d1c190

Although valid syntax in Ruby this is inconsistent with our usage of methods.

Remove memoization for edition

abda3c5

As ActiveRecord handles the memoization for us it seems redundant to have two levels of this in.

Raise an error if retries are exhausted

3051eec

This code would have accidentally been concealing errors.

Add a with_document scope to Edition

e84dd9b

This replaces the many places where we run `joins(:document)`.

Make ignoring substitute unpublishings easier to understand

f81db55

A small refactoring to try make a particularly difficult line of code to follow easier to understand.

Add parenthesis to make precedence clearer

a3153d3

Simpler approach to multiline strings

e636d9e

I think this is easier to follow than the concatentation approach it replaces, still not super easy but seemed less heavy duty than squigly heredoc.

Use exist matchers rather than boolean

045980a

This makes the matcher more specific. It also removes the strings that are output on a failure. We don't use them in other places and in this instance they don't seem particularly useful.

Update DeleteContentItem helper

899016f

For usage with Documents and Editions.

Remove DefaultAttributes concern

159a59f

This is no longer used.

kevindew force-pushed the documents-and-editions branch from 13fa6fe to 159a59f Compare January 26, 2017 10:20

dougdroper approved these changes Jan 26, 2017

View reviewed changes

kevindew merged commit 93e7d0a into master Jan 26, 2017

kevindew deleted the documents-and-editions branch February 2, 2017 14:31

jkempster34 added a commit that referenced this pull request Jul 12, 2023

Fix method name in outdated documentation

0d3dea9

This method was renamed in the past (#683), but the relevant documentation still has the old method name.

jkempster34 mentioned this pull request Jul 12, 2023

Delete placeholder translations #2445

Merged

jkempster34 added a commit that referenced this pull request Jul 12, 2023

Fix method name in outdated documentation

5e5dd69

This method was renamed in the past (#683), but the relevant documentation still has the old method name. Co-authored-by: Cristina <[email protected]>

jkempster34 added a commit that referenced this pull request Jul 13, 2023

Fix method name in outdated documentation

34134d1

This method was renamed in the past (#683), but the relevant documentation still has the old method name. Co-authored-by: Cristina <[email protected]>

CristinaRO added a commit that referenced this pull request Jul 13, 2023

Fix method name in outdated documentation

f77d62a

This method was renamed in the past (#683), but the relevant documentation still has the old method name. Co-authored-by: Cristina <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Switch fully to documents and editions #683

Switch fully to documents and editions #683

thomasleese commented Dec 20, 2016 •

edited

Loading

kevindew commented Dec 20, 2016

binaryberry left a comment

binaryberry Dec 22, 2016

binaryberry Dec 22, 2016

binaryberry Dec 22, 2016

binaryberry Dec 22, 2016

binaryberry Dec 22, 2016

thomasleese Dec 23, 2016

binaryberry Dec 22, 2016

binaryberry Dec 22, 2016

kevindew Jan 3, 2017

kevindew Jan 3, 2017

kevindew Jan 3, 2017

kevindew Jan 3, 2017

kevindew Jan 3, 2017

kevindew Jan 3, 2017

kevindew Jan 3, 2017

kevindew Jan 3, 2017

kevindew Jan 3, 2017

kevindew Jan 3, 2017

kevindew commented Jan 26, 2017

		@@ -112,16 +133,14 @@ def draft_cannot_be_behind_live
		if state == "draft"
		draft_version = user_facing_version

Switch fully to documents and editions #683

Switch fully to documents and editions #683

Conversation

thomasleese commented Dec 20, 2016 • edited Loading

kevindew commented Dec 20, 2016

binaryberry left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kevindew commented Jan 26, 2017

thomasleese commented Dec 20, 2016 •

edited

Loading