Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(gatsby): convert inference-metadata to typescript #24381

Merged
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,72 +1,31 @@
// Tracking structure of nodes to utilize this metadata for schema inference
// Type descriptors stay relevant at any point in time making incremental inference trivial
const { omit } = require(`lodash`)
const {
import { omit } from "lodash"
import {
addNode,
addNodes,
deleteNode,
ignore,
disable,
} = require(`../../schema/infer/inference-metadata`)
const { NodeInterfaceFields } = require(`../../schema/types/node-interface`)
const { typesWithoutInference } = require(`../../schema/types/type-defs`)
} from "../../schema/infer/inference-metadata"
import { NodeInterfaceFields } from "../../schema/types/node-interface"
import { typesWithoutInference } from "../../schema/types/type-defs"

const StepsEnum = {
initialBuild: `initialBuild`,
incrementalBuild: `incrementalBuild`,
}
import { IGatsbyState, ActionsUnion } from "../types"

const initialState = () => {
return {
step: StepsEnum.initialBuild, // `initialBuild` | `incrementalBuild`
typeMap: {},
}
}
const ignoredFields: Set<string> = new Set([
...NodeInterfaceFields,
`__gatsby_resolved`,
])

module.exports = (state = initialState(), action) => {
switch (action.type) {
case `CREATE_NODE`:
case `DELETE_NODE`:
case `DELETE_NODES`:
case `ADD_CHILD_NODE_TO_PARENT_NODE`:
case `ADD_FIELD_TO_NODE`: {
// Perf: disable incremental inference until the first schema build.
// There are plugins which create and delete lots of nodes during bootstrap,
// which makes this reducer to do a lot of unnecessary work.
// Instead we defer the initial metadata creation until the first schema build
// and then enable incremental updates explicitly
if (state.step === StepsEnum.initialBuild) {
return state
}
state.typeMap = incrementalReducer(state.typeMap, action)
return state
}

case `START_INCREMENTAL_INFERENCE`: {
return {
...state,
step: StepsEnum.incrementalBuild,
}
}

case `DELETE_CACHE`: {
return initialState()
}

default: {
state.typeMap = incrementalReducer(state.typeMap, action)
return state
}
}
}

const ignoredFields = new Set([...NodeInterfaceFields, `__gatsby_resolved`])

const initialTypeMetadata = () => {
const initialTypeMetadata = (): { ignoredFields: Set<string> } => {
return { ignoredFields }
}

const incrementalReducer = (state = {}, action) => {
const incrementalReducer = (
state: IGatsbyState["inferenceMetadata"]["typeMap"] = {},
action: ActionsUnion
): IGatsbyState["inferenceMetadata"]["typeMap"] => {
switch (action.type) {
case `CREATE_TYPES`: {
const typeDefs = Array.isArray(action.payload)
Expand Down Expand Up @@ -123,8 +82,8 @@ const incrementalReducer = (state = {}, action) => {
// Can't simply add { fields: { [addedField]: node.fields[addedField] } }
// because it will count `fields` key twice for the same node
const previousFields = omit(node.fields, [addedField])
state[type] = deleteNode(state[type], { fields: previousFields })
state[type] = addNode(state[type], { fields: node.fields })
state[type] = deleteNode(state[type], { ...node, fields: previousFields })
state[type] = addNode(state[type], { ...node, fields: node.fields })
Copy link
Contributor Author

@Kornil Kornil May 23, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These 2 changes have the potential to affect production code as we are now sending a full node instead of just fields. I based this on the fact that the action themselves (deleteNode and addNode) are already typed and coded to accept a full node as second parameter.

Please let me know if this change is acceptable or how I can write it better.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels correct, so it's just a matter of if tests will pass correctly with this change. Thanks for figuring this out!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests seem to pass nicely so it should be all good, I just wanted to make sure this change was seen.


// TODO: there might be an edge case when the same field is "added" twice.
// Then we'll count it twice in metadata. The only way to avoid it as I see it
Expand Down Expand Up @@ -161,3 +120,55 @@ const incrementalReducer = (state = {}, action) => {
return state
}
}

const StepsEnum = {
initialBuild: `initialBuild`,
incrementalBuild: `incrementalBuild`,
}
blainekasten marked this conversation as resolved.
Show resolved Hide resolved

const initialState = (): IGatsbyState["inferenceMetadata"] => {
return {
step: StepsEnum.initialBuild, // `initialBuild` | `incrementalBuild`
typeMap: {},
}
}

module.exports = (
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be a named export and update the import reference.

e.g.

export function inferenceMetadataReducer(state, action) { .... }

state: IGatsbyState["inferenceMetadata"] = initialState(),
action: ActionsUnion
): IGatsbyState["inferenceMetadata"] => {
switch (action.type) {
case `CREATE_NODE`:
case `DELETE_NODE`:
case `DELETE_NODES`:
case `ADD_CHILD_NODE_TO_PARENT_NODE`:
case `ADD_FIELD_TO_NODE`: {
// Perf: disable incremental inference until the first schema build.
// There are plugins which create and delete lots of nodes during bootstrap,
// which makes this reducer to do a lot of unnecessary work.
// Instead we defer the initial metadata creation until the first schema build
// and then enable incremental updates explicitly
if (state.step === StepsEnum.initialBuild) {
return state
}
state.typeMap = incrementalReducer(state.typeMap, action)
return state
}

case `START_INCREMENTAL_INFERENCE`: {
return {
...state,
step: StepsEnum.incrementalBuild,
}
}

case `DELETE_CACHE`: {
return initialState()
}

default: {
state.typeMap = incrementalReducer(state.typeMap, action)
return state
}
}
}
36 changes: 27 additions & 9 deletions packages/gatsby/src/redux/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import { DocumentNode, GraphQLSchema } from "graphql"
import { SchemaComposer } from "graphql-compose"
import { IGatsbyCLIState } from "gatsby-cli/src/reporter/redux/types"
import { InternalJobInterface, JobResultInterface } from "../utils/jobs-manager"
import { ITypeMetadata } from "../schema/infer/inference-metadata"

type SystemPath = string
type Identifier = string
Expand Down Expand Up @@ -75,6 +76,7 @@ export interface IGatsbyNode {
}
__gatsby_resolved: any // TODO
[key: string]: unknown
fields: string[]
}

export interface IGatsbyPlugin {
Expand Down Expand Up @@ -223,12 +225,7 @@ export interface IGatsbyState {
inferenceMetadata: {
step: string // TODO make enum or union
typeMap: {
[key: string]: {
ignoredFields: Set<string>
total: number
dirty: boolean
fieldMap: any // TODO
}
[key: string]: ITypeMetadata
}
}
pageDataStats: Map<SystemPath, number>
Expand Down Expand Up @@ -291,6 +288,9 @@ export type ActionsUnion =
| ICreateJobAction
| ISetJobAction
| IEndJobAction
| IStartIncrementalInferenceAction
| IBuildTypeMetadataAction
| IDisableTypeInferenceAction

interface ISetBabelPluginAction {
type: `SET_BABEL_PLUGIN`
Expand Down Expand Up @@ -603,11 +603,13 @@ export interface ISetSiteConfig {
export interface ICreateNodeAction {
type: `CREATE_NODE`
payload: IGatsbyNode
oldNode?: IGatsbyNode
}

export interface IAddFieldToNodeAction {
type: `ADD_FIELD_TO_NODE`
payload: IGatsbyNode
addedField: string
}

export interface IAddChildNodeToParentNodeAction {
Expand All @@ -617,14 +619,13 @@ export interface IAddChildNodeToParentNodeAction {

export interface IDeleteNodeAction {
type: `DELETE_NODE`
payload: {
id: Identifier
}
payload: IGatsbyNode
}

export interface IDeleteNodesAction {
type: `DELETE_NODES`
payload: Identifier[]
fullNodes: IGatsbyNode[]
}

export interface IAddPageDataStatsAction {
Expand All @@ -639,3 +640,20 @@ export interface ITouchNodeAction {
type: `TOUCH_NODE`
payload: Identifier
}

interface IStartIncrementalInferenceAction {
type: `START_INCREMENTAL_INFERENCE`
}

interface IBuildTypeMetadataAction {
type: `BUILD_TYPE_METADATA`
payload: {
nodes: IGatsbyNode[]
typeName: string
}
}

interface IDisableTypeInferenceAction {
type: `DISABLE_TYPE_INFERENCE`
payload: string[]
}
2 changes: 1 addition & 1 deletion packages/gatsby/src/schema/infer/inference-metadata.ts
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ export interface ITypeMetadata {
dirty?: boolean
total?: number
ignoredFields?: Set<string>
fieldMap: Record<string, IValueDescriptor>
fieldMap?: Record<string, IValueDescriptor>
typeConflictReporter?: TypeConflictReporter
[key: string]: unknown
}
Expand Down