Skip to content

Commit

Permalink
🪟🔧 Connector builder: Performance improvements (#20620)
Browse files Browse the repository at this point in the history
* improve some types

* improve further

* clean up a bit more

* refactor loading state

* move loading state up

* remove isLoading references

* remove unused props and make fetch connector error work

* remove special component for name

* remove top level state for unifinished flows

* start removing uiwidget

* Update airbyte-webapp/src/views/Connector/ConnectorCard/ConnectorCard.module.scss

Co-authored-by: Tim Roes <[email protected]>

* remove undefined option for selected id

* remove unused prop

* fix types

* remove uiwidget state

* clean up

* adjust comment

* handle errors in a nice way

* do not respect default on oneOf fields

* rename to formblock

* reduce re-renders

* pass error to secure inputs

* simplify and improve styling

* align top

* code review

* remove comment

* review comments

* rename file

* be strict about boolean values

* add example

* track form error in error boundary

* review comments

* handle unexpected cases better

* speed up some bits

* more changes

* enrich error with connector id

* 🪟🎉 Add copy stream button (#20577)

* add copy stream button

* review comments

* rename prop

* 🪟🎉 Connector builder: Integrate connector form for test input (#20385)

* move connector builder components into the same shared components/connectorBuilder directory

* move diff over from poc branch

* save current progress

* add modal for adding streams

* focus stream after adding and reset button style

* add reset confirm modal and select view on add

* style global config and streams buttons

* styling improvements

* handle long stream names better

* pull in connector manifest schema directly

* add box shadows to resizable panels

* upgrade orval and use connector manifest schema directly

* remove airbyte protocol from connector builder api spec

* generate python models from openapi change

* fix position of yaml toggle

* handle no stream case with better looking message

* group global fields into single object and fix console error

* confirmation modal on toggling dirty form + cleanup

* fix connector name display

* undo change to manifest schema

* remove commented code

* remove unnecessary change

* fix spacing

* use shadow mixin for connector img

* add comment about connector img

* change onSubmit to no-op

* remove console log

* clean up styling

* simplify sidebar to remove StreamSelectButton component

* swap colors of toggle

* move FormikPatch to src/core/form

* move types up to connectorBuilder/ level

* use grid display for ui yaml toggle button

* use spread instead of setting array index directly

* add intl in missing places

* pull connector manifest schema in through separate openapi spec

* use correct intl string id

* throttle setting json manifest in yaml editor

* use  button prop instead of manually styling

* consolidate AddStreamButton styles

* fix sidebar flex styles

* use specific flex properties instead of flex

* clean up download and reset button styles

* use row-reverse for yaml editor download button

* fix stream selector styles to remove margins

* give connector setup guide panel same corner and shadow styles

* remove blur from page display

* set view to stream when selected in test panel

* add placeholder when stream name is empty

* switch to index-based stream selection to preserve testing panel selected stream on rename

* handle empty name in stream selector

* make connector form work in connector builder

* fix small stuff

* add warning label

* review comments

* adjust translation

Co-authored-by: lmossman <[email protected]>

* use request_body_json instead of request_body_data

* 🪟 🎨 Move `Add` button into the line of Connector Builder key value list fields (#20699)

* move add button into line

* add stories for empty with control, and content + control

* change button name to Control

* 🪟🎉 Connector builder: Allow defining inputs (#20431)

* move connector builder components into the same shared components/connectorBuilder directory

* move diff over from poc branch

* save current progress

* add modal for adding streams

* focus stream after adding and reset button style

* add reset confirm modal and select view on add

* style global config and streams buttons

* styling improvements

* handle long stream names better

* pull in connector manifest schema directly

* add box shadows to resizable panels

* upgrade orval and use connector manifest schema directly

* remove airbyte protocol from connector builder api spec

* generate python models from openapi change

* fix position of yaml toggle

* handle no stream case with better looking message

* group global fields into single object and fix console error

* confirmation modal on toggling dirty form + cleanup

* fix connector name display

* undo change to manifest schema

* remove commented code

* remove unnecessary change

* fix spacing

* use shadow mixin for connector img

* add comment about connector img

* change onSubmit to no-op

* remove console log

* clean up styling

* simplify sidebar to remove StreamSelectButton component

* swap colors of toggle

* move FormikPatch to src/core/form

* move types up to connectorBuilder/ level

* use grid display for ui yaml toggle button

* use spread instead of setting array index directly

* add intl in missing places

* pull connector manifest schema in through separate openapi spec

* use correct intl string id

* throttle setting json manifest in yaml editor

* use  button prop instead of manually styling

* consolidate AddStreamButton styles

* fix sidebar flex styles

* use specific flex properties instead of flex

* clean up download and reset button styles

* use row-reverse for yaml editor download button

* fix stream selector styles to remove margins

* give connector setup guide panel same corner and shadow styles

* remove blur from page display

* set view to stream when selected in test panel

* add placeholder when stream name is empty

* switch to index-based stream selection to preserve testing panel selected stream on rename

* handle empty name in stream selector

* make connector form work in connector builder

* wip

* fix small stuff

* add basic input UI

* user inputs

* make most of inputs configuration work

* fix a bunch of stuff

* handle unknown config types

* add warning label

* fix label

* fix some styling

* review comments

* improve state management and error handling

* handle stored form values that don't contain new fields properly

* Update airbyte-webapp/src/locales/en.json

Co-authored-by: Lake Mossman <[email protected]>

* Update airbyte-webapp/src/components/connectorBuilder/Builder/InputsView.tsx

Co-authored-by: Lake Mossman <[email protected]>

* inputs editing weirdness

* input form reset

* using the Label component

* 🪟🎉 Connector builder authentication (#20645)

* allow auth configuration

* check for conflicts with the inferred inputs

* fix invisible inputs

* reduce redundancy and hide advanced input options for inferred inputs

* unnecessary validation

* typo

* unnecessary effect hook

* build spec even for invalid forms but do not update stream list

* fix keys

* 🪟🎉 Connector builder: Session token and oauth authentication (#20712)

* session token and oauth authentication

* fill in session token variable

* typos

* make sure validation error does not go away

* 🪟🎉 Connector builder: Always validate inputs form (#20664)

* validate user input outside of form

* review comments

Co-authored-by: lmossman <[email protected]>

Co-authored-by: lmossman <[email protected]>

* fix merge conflict with dropdown prop being renamed to control

* [Connector Builder] Add paginator (#20698)

* move connector builder components into the same shared components/connectorBuilder directory

* move diff over from poc branch

* save current progress

* add modal for adding streams

* focus stream after adding and reset button style

* add reset confirm modal and select view on add

* style global config and streams buttons

* styling improvements

* handle long stream names better

* pull in connector manifest schema directly

* add box shadows to resizable panels

* upgrade orval and use connector manifest schema directly

* remove airbyte protocol from connector builder api spec

* generate python models from openapi change

* fix position of yaml toggle

* handle no stream case with better looking message

* group global fields into single object and fix console error

* confirmation modal on toggling dirty form + cleanup

* fix connector name display

* undo change to manifest schema

* remove commented code

* remove unnecessary change

* fix spacing

* use shadow mixin for connector img

* add comment about connector img

* change onSubmit to no-op

* remove console log

* clean up styling

* simplify sidebar to remove StreamSelectButton component

* swap colors of toggle

* move FormikPatch to src/core/form

* move types up to connectorBuilder/ level

* use grid display for ui yaml toggle button

* use spread instead of setting array index directly

* add intl in missing places

* pull connector manifest schema in through separate openapi spec

* use correct intl string id

* throttle setting json manifest in yaml editor

* use  button prop instead of manually styling

* consolidate AddStreamButton styles

* fix sidebar flex styles

* use specific flex properties instead of flex

* clean up download and reset button styles

* use row-reverse for yaml editor download button

* fix stream selector styles to remove margins

* give connector setup guide panel same corner and shadow styles

* remove blur from page display

* set view to stream when selected in test panel

* add placeholder when stream name is empty

* switch to index-based stream selection to preserve testing panel selected stream on rename

* handle empty name in stream selector

* make connector form work in connector builder

* wip

* fix small stuff

* add basic input UI

* user inputs

* make most of inputs configuration work

* fix a bunch of stuff

* handle unknown config types

* add warning label

* fix label

* fix some styling

* review comments

* improve state management and error handling

* allow auth configuration

* check for conflicts with the inferred inputs

* fix invisible inputs

* handle stored form values that don't contain new fields properly

* session token and oauth authentication

* fill in session token variable

* fix merge of default values

* add primaryKey and cursorField to builder types, and consolidate default valeues to types.ts

* add cursor and primary key fields to ui

* save

* add page size and token option inputs

* fixes after rebase

* add pagination

* fix pagination types

* handle empty field_name better

* Update airbyte-webapp/src/locales/en.json

Co-authored-by: Lake Mossman <[email protected]>

* Update airbyte-webapp/src/components/connectorBuilder/Builder/InputsView.tsx

Co-authored-by: Lake Mossman <[email protected]>

* inputs editing weirdness

* input form reset

* using the Label component

* reduce redundancy and hide advanced input options for inferred inputs

* unnecessary validation

* typo

* unnecessary effect hook

* build spec even for invalid forms but do not update stream list

* typos

* make sure validation error does not go away

* make primary key and cursor optional, and reorder

* save toggle group progress

* fix style of toggle label

* handle empty values better

* fix page size/token option field validation and rendering

* handle cursor pagination page size option correctly

Co-authored-by: Joe Reuter <[email protected]>

* [Connector Builder] Add stream slicer (#20748)

* move connector builder components into the same shared components/connectorBuilder directory

* move diff over from poc branch

* save current progress

* add modal for adding streams

* focus stream after adding and reset button style

* add reset confirm modal and select view on add

* style global config and streams buttons

* styling improvements

* handle long stream names better

* pull in connector manifest schema directly

* add box shadows to resizable panels

* upgrade orval and use connector manifest schema directly

* remove airbyte protocol from connector builder api spec

* generate python models from openapi change

* fix position of yaml toggle

* handle no stream case with better looking message

* group global fields into single object and fix console error

* confirmation modal on toggling dirty form + cleanup

* fix connector name display

* undo change to manifest schema

* remove commented code

* remove unnecessary change

* fix spacing

* use shadow mixin for connector img

* add comment about connector img

* change onSubmit to no-op

* remove console log

* clean up styling

* simplify sidebar to remove StreamSelectButton component

* swap colors of toggle

* move FormikPatch to src/core/form

* move types up to connectorBuilder/ level

* use grid display for ui yaml toggle button

* use spread instead of setting array index directly

* add intl in missing places

* pull connector manifest schema in through separate openapi spec

* use correct intl string id

* throttle setting json manifest in yaml editor

* use  button prop instead of manually styling

* consolidate AddStreamButton styles

* fix sidebar flex styles

* use specific flex properties instead of flex

* clean up download and reset button styles

* use row-reverse for yaml editor download button

* fix stream selector styles to remove margins

* give connector setup guide panel same corner and shadow styles

* remove blur from page display

* set view to stream when selected in test panel

* add placeholder when stream name is empty

* switch to index-based stream selection to preserve testing panel selected stream on rename

* handle empty name in stream selector

* make connector form work in connector builder

* wip

* fix small stuff

* add basic input UI

* user inputs

* make most of inputs configuration work

* fix a bunch of stuff

* handle unknown config types

* add warning label

* fix label

* fix some styling

* review comments

* improve state management and error handling

* allow auth configuration

* check for conflicts with the inferred inputs

* fix invisible inputs

* handle stored form values that don't contain new fields properly

* session token and oauth authentication

* fill in session token variable

* fix merge of default values

* add primaryKey and cursorField to builder types, and consolidate default valeues to types.ts

* add cursor and primary key fields to ui

* save

* add page size and token option inputs

* fixes after rebase

* add pagination

* fix pagination types

* handle empty field_name better

* Update airbyte-webapp/src/locales/en.json

Co-authored-by: Lake Mossman <[email protected]>

* Update airbyte-webapp/src/components/connectorBuilder/Builder/InputsView.tsx

Co-authored-by: Lake Mossman <[email protected]>

* inputs editing weirdness

* input form reset

* using the Label component

* reduce redundancy and hide advanced input options for inferred inputs

* unnecessary validation

* typo

* unnecessary effect hook

* build spec even for invalid forms but do not update stream list

* typos

* make sure validation error does not go away

* make primary key and cursor optional, and reorder

* save toggle group progress

* fix style of toggle label

* handle empty values better

* fix page size/token option field validation and rendering

* handle cursor pagination page size option correctly

* save stream slicer progress

* finish stream slicer

* fix stream slicer fields and validation

Co-authored-by: Joe Reuter <[email protected]>

* debounce form builder values update to reduce load

* 🪟🔧  Connector builder: use new lowcode manifest (#20715)

* use new manifest yaml

* Update airbyte-webapp/src/components/connectorBuilder/types.ts

Co-authored-by: Lake Mossman <[email protected]>

* use updated manifest types

Co-authored-by: Lake Mossman <[email protected]>

* debounce validation as well

* akways show stream test button in error state if there are errors

* fix type of oauth input

* review comments

* fix more

* add validation schema for add stream form

* validate all views on test click

* add type to prevent console warning

* review comment

* make sure testing state and form state stay consistent

* improve builder errors

* remove test state from streamconfig view

* remove console log

* remove unnecessary positive index check

Co-authored-by: Tim Roes <[email protected]>
Co-authored-by: lmossman <[email protected]>
  • Loading branch information
3 people authored Jan 5, 2023
1 parent db3d398 commit 6c08f43
Show file tree
Hide file tree
Showing 13 changed files with 232 additions and 158 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ import { Form } from "formik";
import debounce from "lodash/debounce";
import { useEffect, useMemo } from "react";

import { BuilderView, useConnectorBuilderState } from "services/connectorBuilder/ConnectorBuilderStateService";
import { BuilderView, useConnectorBuilderFormState } from "services/connectorBuilder/ConnectorBuilderStateService";

import { builderFormValidationSchema, BuilderFormValues } from "../types";
import styles from "./Builder.module.scss";
Expand All @@ -29,7 +29,7 @@ function getView(selectedView: BuilderView, hasMultipleStreams: boolean) {
}

export const Builder: React.FC<BuilderProps> = ({ values, toggleYamlEditor, validateForm }) => {
const { setBuilderFormValues, selectedView } = useConnectorBuilderState();
const { setBuilderFormValues, selectedView } = useConnectorBuilderFormState();
const debouncedSetBuilderFormValues = useMemo(
() =>
debounce((values) => {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import { faSliders, faUser } from "@fortawesome/free-solid-svg-icons";
import { FontAwesomeIcon } from "@fortawesome/react-fontawesome";
import classnames from "classnames";
import { useFormikContext } from "formik";
import React from "react";
import { FormattedMessage, useIntl } from "react-intl";

import Indicator from "components/Indicator";
Expand All @@ -10,7 +11,7 @@ import { Heading } from "components/ui/Heading";
import { Text } from "components/ui/Text";

import { useConfirmationModalService } from "hooks/services/ConfirmationModal";
import { BuilderView, useConnectorBuilderState } from "services/connectorBuilder/ConnectorBuilderStateService";
import { BuilderView, useConnectorBuilderFormState } from "services/connectorBuilder/ConnectorBuilderStateService";

import { DownloadYamlButton } from "../DownloadYamlButton";
import { BuilderFormValues, DEFAULT_BUILDER_FORM_VALUES, getInferredInputs } from "../types";
Expand Down Expand Up @@ -52,11 +53,11 @@ interface BuilderSidebarProps {
toggleYamlEditor: () => void;
}

export const BuilderSidebar: React.FC<BuilderSidebarProps> = ({ className, toggleYamlEditor }) => {
export const BuilderSidebar: React.FC<BuilderSidebarProps> = React.memo(({ className, toggleYamlEditor }) => {
const { formatMessage } = useIntl();
const { hasErrors } = useBuilderErrors();
const { openConfirmationModal, closeConfirmationModal } = useConfirmationModalService();
const { yamlManifest, selectedView, setSelectedView, setTestStreamIndex } = useConnectorBuilderState();
const { yamlManifest, selectedView, setSelectedView } = useConnectorBuilderFormState();
const { values, setValues } = useFormikContext<BuilderFormValues>();
const handleResetForm = () => {
openConfirmationModal({
Expand All @@ -72,9 +73,6 @@ export const BuilderSidebar: React.FC<BuilderSidebarProps> = ({ className, toggl
};
const handleViewSelect = (selectedView: BuilderView) => {
setSelectedView(selectedView);
if (selectedView !== "global" && selectedView !== "inputs") {
setTestStreamIndex(selectedView);
}
};

return (
Expand Down Expand Up @@ -150,4 +148,4 @@ export const BuilderSidebar: React.FC<BuilderSidebarProps> = ({ className, toggl
</Button>
</div>
);
};
});
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import { CodeEditor } from "components/ui/CodeEditor";
import { Text } from "components/ui/Text";

import { useConfirmationModalService } from "hooks/services/ConfirmationModal";
import { BuilderView, useConnectorBuilderState } from "services/connectorBuilder/ConnectorBuilderStateService";
import { BuilderView, useConnectorBuilderFormState } from "services/connectorBuilder/ConnectorBuilderStateService";

import { BuilderStream } from "../types";
import { AddStreamButton } from "./AddStreamButton";
Expand All @@ -33,7 +33,7 @@ export const StreamConfigView: React.FC<StreamConfigViewProps> = ({ streamNum, h
const [field, , helpers] = useField<BuilderStream[]>("streams");
const [selectedTab, setSelectedTab] = useState<"configuration" | "schema">("configuration");
const { openConfirmationModal, closeConfirmationModal } = useConfirmationModalService();
const { setSelectedView, setTestStreamIndex } = useConnectorBuilderState();
const { setSelectedView } = useConnectorBuilderFormState();

const streamPath = `streams[${streamNum}]`;
const streamFieldPath = (fieldPath: string) => `${streamPath}.${fieldPath}`;
Expand All @@ -49,7 +49,6 @@ export const StreamConfigView: React.FC<StreamConfigViewProps> = ({ streamNum, h
const viewToSelect: BuilderView = updatedStreams.length === 0 ? "global" : streamToSelect;
helpers.setValue(updatedStreams);
setSelectedView(viewToSelect);
setTestStreamIndex(streamToSelect);
closeConfirmationModal();
},
});
Expand Down Expand Up @@ -80,7 +79,6 @@ export const StreamConfigView: React.FC<StreamConfigViewProps> = ({ streamNum, h
<AddStreamButton
onAddStream={(addedStreamNum) => {
setSelectedView(addedStreamNum);
setTestStreamIndex(addedStreamNum);
}}
initialValues={field.value[streamNum]}
button={
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import { FormattedMessage } from "react-intl";
import { Button } from "components/ui/Button";
import { Tooltip } from "components/ui/Tooltip";

import { useConnectorBuilderState } from "services/connectorBuilder/ConnectorBuilderStateService";
import { useConnectorBuilderFormState } from "services/connectorBuilder/ConnectorBuilderStateService";
import { downloadFile } from "utils/file";

import styles from "./DownloadYamlButton.module.scss";
Expand All @@ -18,7 +18,7 @@ interface DownloadYamlButtonProps {
}

export const DownloadYamlButton: React.FC<DownloadYamlButtonProps> = ({ className, yaml, yamlIsValid }) => {
const { editorView } = useConnectorBuilderState();
const { editorView } = useConnectorBuilderFormState();
const { hasErrors, validateAndTouch } = useBuilderErrors();

const downloadYaml = () => {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,22 +12,25 @@ import { Tooltip } from "components/ui/Tooltip";

import { SourceDefinitionSpecificationDraft } from "core/domain/connector";
import { StreamReadRequestBodyConfig } from "core/request/ConnectorBuilderClient";
import { useConnectorBuilderState } from "services/connectorBuilder/ConnectorBuilderStateService";
import { useConnectorBuilderTestState } from "services/connectorBuilder/ConnectorBuilderStateService";
import { useConnectorBuilderFormState } from "services/connectorBuilder/ConnectorBuilderStateService";
import { ConnectorForm } from "views/Connector/ConnectorForm";

import styles from "./ConfigMenu.module.scss";
import { ConfigMenuErrorBoundaryComponent } from "./ConfigMenuErrorBoundary";

interface ConfigMenuProps {
className?: string;
configJsonErrors: number;
testInputJsonErrors: number;
isOpen: boolean;
setIsOpen: (open: boolean) => void;
}

export const ConfigMenu: React.FC<ConfigMenuProps> = ({ className, configJsonErrors, isOpen, setIsOpen }) => {
export const ConfigMenu: React.FC<ConfigMenuProps> = ({ className, testInputJsonErrors, isOpen, setIsOpen }) => {
const { formatMessage } = useIntl();
const { configJson, setConfigJson, jsonManifest, editorView, setEditorView } = useConnectorBuilderState();
const { jsonManifest, editorView, setEditorView } = useConnectorBuilderFormState();

const { testInputJson, setTestInputJson } = useConnectorBuilderTestState();

const [showInputsWarning, setShowInputsWarning] = useLocalStorage<boolean>("connectorBuilderInputsWarning", true);

Expand Down Expand Up @@ -64,8 +67,8 @@ export const ConfigMenu: React.FC<ConfigMenuProps> = ({ className, configJsonErr
>
<FormattedMessage id="connectorBuilder.inputsButton" />
</Button>
{configJsonErrors > 0 && (
<NumberBadge className={styles.inputsErrorBadge} value={configJsonErrors} color="red" />
{testInputJsonErrors > 0 && (
<NumberBadge className={styles.inputsErrorBadge} value={testInputJsonErrors} color="red" />
)}
</>
}
Expand Down Expand Up @@ -110,16 +113,16 @@ export const ConfigMenu: React.FC<ConfigMenuProps> = ({ className, configJsonErr
bodyClassName={styles.formContent}
footerClassName={styles.inputFormModalFooter}
selectedConnectorDefinitionSpecification={connectorDefinitionSpecification}
formValues={{ connectionConfiguration: configJson }}
formValues={{ connectionConfiguration: testInputJson }}
onSubmit={async (values) => {
setConfigJson(values.connectionConfiguration as StreamReadRequestBodyConfig);
setTestInputJson(values.connectionConfiguration as StreamReadRequestBodyConfig);
setIsOpen(false);
}}
onCancel={() => {
setIsOpen(false);
}}
onReset={() => {
setConfigJson({});
setTestInputJson({});
}}
submitLabel={formatMessage({ id: "connectorBuilder.saveInputsForm" })}
/>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,10 @@ import { useIntl } from "react-intl";
import { Heading } from "components/ui/Heading";
import { ListBox, ListBoxControlButtonProps } from "components/ui/ListBox";

import { useConnectorBuilderState } from "services/connectorBuilder/ConnectorBuilderStateService";
import {
useConnectorBuilderTestState,
useConnectorBuilderFormState,
} from "services/connectorBuilder/ConnectorBuilderStateService";

import { ReactComponent as CaretDownIcon } from "../../ui/ListBox/CaretDownIcon.svg";
import styles from "./StreamSelector.module.scss";
Expand All @@ -27,7 +30,8 @@ const ControlButton: React.FC<ListBoxControlButtonProps<string>> = ({ selectedOp

export const StreamSelector: React.FC<StreamSelectorProps> = ({ className }) => {
const { formatMessage } = useIntl();
const { streams, selectedView, testStreamIndex, setSelectedView, setTestStreamIndex } = useConnectorBuilderState();
const { selectedView, setSelectedView } = useConnectorBuilderFormState();
const { streams, testStreamIndex, setTestStreamIndex } = useConnectorBuilderTestState();
const options = streams.map((stream) => {
const label =
stream.name && stream.name.trim() ? capitalize(stream.name) : formatMessage({ id: "connectorBuilder.emptyName" });
Expand All @@ -39,7 +43,7 @@ export const StreamSelector: React.FC<StreamSelectorProps> = ({ className }) =>
if (selectedStreamIndex >= 0) {
setTestStreamIndex(selectedStreamIndex);

if (selectedView !== "global" && selectedStreamIndex >= 0) {
if (selectedView !== "global") {
setSelectedView(selectedStreamIndex);
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,27 +7,27 @@ import { Button } from "components/ui/Button";
import { Text } from "components/ui/Text";
import { Tooltip } from "components/ui/Tooltip";

import { useConnectorBuilderState } from "services/connectorBuilder/ConnectorBuilderStateService";
import { useConnectorBuilderFormState } from "services/connectorBuilder/ConnectorBuilderStateService";

import { useBuilderErrors } from "../useBuilderErrors";
import styles from "./StreamTestButton.module.scss";

interface StreamTestButtonProps {
readStream: () => void;
hasConfigJsonErrors: boolean;
hasTestInputJsonErrors: boolean;
setTestInputOpen: (open: boolean) => void;
}

export const StreamTestButton: React.FC<StreamTestButtonProps> = ({
readStream,
hasConfigJsonErrors,
hasTestInputJsonErrors,
setTestInputOpen,
}) => {
const { editorView, yamlIsValid } = useConnectorBuilderState();
const { editorView, yamlIsValid } = useConnectorBuilderFormState();
const { hasErrors, validateAndTouch } = useBuilderErrors();

const handleClick = () => {
if (hasConfigJsonErrors) {
if (hasTestInputJsonErrors) {
setTestInputOpen(true);
return;
}
Expand All @@ -49,7 +49,7 @@ export const StreamTestButton: React.FC<StreamTestButtonProps> = ({
tooltipContent = <FormattedMessage id="connectorBuilder.invalidYamlTest" />;
}

if ((editorView === "ui" && hasErrors(false)) || hasConfigJsonErrors) {
if ((editorView === "ui" && hasErrors(false)) || hasTestInputJsonErrors) {
showWarningIcon = true;
tooltipContent = <FormattedMessage id="connectorBuilder.configErrorsTest" />;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,19 +6,23 @@ import { Spinner } from "components/ui/Spinner";
import { Text } from "components/ui/Text";

import { useReadStream } from "services/connectorBuilder/ConnectorBuilderApiService";
import { useConnectorBuilderState } from "services/connectorBuilder/ConnectorBuilderStateService";
import {
useConnectorBuilderTestState,
useConnectorBuilderFormState,
} from "services/connectorBuilder/ConnectorBuilderStateService";

import { LogsDisplay } from "./LogsDisplay";
import { ResultDisplay } from "./ResultDisplay";
import { StreamTestButton } from "./StreamTestButton";
import styles from "./StreamTester.module.scss";

export const StreamTester: React.FC<{
hasConfigJsonErrors: boolean;
hasTestInputJsonErrors: boolean;
setTestInputOpen: (open: boolean) => void;
}> = ({ hasConfigJsonErrors, setTestInputOpen }) => {
}> = ({ hasTestInputJsonErrors, setTestInputOpen }) => {
const { formatMessage } = useIntl();
const { jsonManifest, configJson, streams, testStreamIndex } = useConnectorBuilderState();
const { jsonManifest } = useConnectorBuilderFormState();
const { streams, testInputJson, testStreamIndex } = useConnectorBuilderTestState();
const {
data: streamReadData,
refetch: readStream,
Expand All @@ -28,7 +32,7 @@ export const StreamTester: React.FC<{
} = useReadStream({
manifest: jsonManifest,
stream: streams[testStreamIndex]?.name,
config: configJson,
config: testInputJson,
});

const [logsFlex, setLogsFlex] = useState(0);
Expand Down Expand Up @@ -60,7 +64,7 @@ export const StreamTester: React.FC<{

<StreamTestButton
readStream={readStream}
hasConfigJsonErrors={hasConfigJsonErrors}
hasTestInputJsonErrors={hasTestInputJsonErrors}
setTestInputOpen={setTestInputOpen}
/>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,10 @@ import { jsonSchemaToFormBlock } from "core/form/schemaToFormBlock";
import { buildYupFormForJsonSchema } from "core/form/schemaToYup";
import { StreamReadRequestBodyConfig } from "core/request/ConnectorBuilderClient";
import { Spec } from "core/request/ConnectorManifest";
import { useConnectorBuilderState } from "services/connectorBuilder/ConnectorBuilderStateService";
import {
useConnectorBuilderTestState,
useConnectorBuilderFormState,
} from "services/connectorBuilder/ConnectorBuilderStateService";
import { links } from "utils/links";

import { ConfigMenu } from "./ConfigMenu";
Expand All @@ -20,28 +23,29 @@ import styles from "./StreamTestingPanel.module.scss";

const EMPTY_SCHEMA = {};

function useConfigJsonErrors(configJson: StreamReadRequestBodyConfig, spec?: Spec): number {
function useTestInputJsonErrors(testInputJson: StreamReadRequestBodyConfig, spec?: Spec): number {
return useMemo(() => {
try {
const jsonSchema = spec && spec.connection_specification ? spec.connection_specification : EMPTY_SCHEMA;
const formFields = jsonSchemaToFormBlock(jsonSchema);
const validationSchema = buildYupFormForJsonSchema(jsonSchema, formFields);
validationSchema.validateSync(configJson, { abortEarly: false });
validationSchema.validateSync(testInputJson, { abortEarly: false });
return 0;
} catch (e) {
if (ValidationError.isError(e)) {
return e.errors.length;
}
return 1;
}
}, [configJson, spec]);
}, [testInputJson, spec]);
}

export const StreamTestingPanel: React.FC<unknown> = () => {
const [isTestInputOpen, setTestInputOpen] = useState(false);
const { jsonManifest, configJson, streamListErrorMessage, yamlEditorIsMounted } = useConnectorBuilderState();
const { jsonManifest, yamlEditorIsMounted } = useConnectorBuilderFormState();
const { testInputJson, streamListErrorMessage } = useConnectorBuilderTestState();

const configJsonErrors = useConfigJsonErrors(configJson, jsonManifest.spec);
const testInputJsonErrors = useTestInputJsonErrors(testInputJson, jsonManifest.spec);

if (!yamlEditorIsMounted) {
return (
Expand All @@ -57,7 +61,7 @@ export const StreamTestingPanel: React.FC<unknown> = () => {
<div className={styles.container}>
<ConfigMenu
className={styles.configButton}
configJsonErrors={configJsonErrors}
testInputJsonErrors={testInputJsonErrors}
isOpen={isTestInputOpen}
setIsOpen={setTestInputOpen}
/>
Expand All @@ -72,7 +76,7 @@ export const StreamTestingPanel: React.FC<unknown> = () => {
{hasStreams && streamListErrorMessage === undefined && (
<div className={styles.selectAndTestContainer}>
<StreamSelector className={styles.streamSelector} />
<StreamTester hasConfigJsonErrors={configJsonErrors > 0} setTestInputOpen={setTestInputOpen} />
<StreamTester hasTestInputJsonErrors={testInputJsonErrors > 0} setTestInputOpen={setTestInputOpen} />
</div>
)}
{hasStreams && streamListErrorMessage !== undefined && (
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import { CodeEditor } from "components/ui/CodeEditor";

import { ConnectorManifest } from "core/request/ConnectorManifest";
import { useConfirmationModalService } from "hooks/services/ConfirmationModal";
import { useConnectorBuilderState } from "services/connectorBuilder/ConnectorBuilderStateService";
import { useConnectorBuilderFormState } from "services/connectorBuilder/ConnectorBuilderStateService";

import { UiYamlToggleButton } from "../Builder/UiYamlToggleButton";
import { DownloadYamlButton } from "../DownloadYamlButton";
Expand All @@ -31,7 +31,7 @@ export const YamlEditor: React.FC<YamlEditorProps> = ({ toggleYamlEditor }) => {
setYamlEditorIsMounted,
setYamlIsValid,
setJsonManifest,
} = useConnectorBuilderState();
} = useConnectorBuilderFormState();
const [yamlValue, setYamlValue] = useState(yamlManifest);

// debounce the setJsonManifest calls so that it doesnt result in a network call for every keystroke
Expand Down
Loading

0 comments on commit 6c08f43

Please sign in to comment.