Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add forecasting dashboard - UI - transformation creation, table, and comparison #1874

Merged
merged 50 commits into from
Jan 13, 2023
Merged
Show file tree
Hide file tree
Changes from 47 commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
6f16591
Add postga build trigger (#1755) (#1756)
gaugup Sep 29, 2022
a54a531
Merge branch 'main' into postga
gaugup Oct 3, 2022
7bb4087
Add model wrapper for wrapping predictions and test data (#1762)
gaugup Oct 5, 2022
8395b89
Merge branch 'main' into postga
gaugup Oct 7, 2022
c400c30
Merge branch 'main' into postga
gaugup Oct 7, 2022
d0d338e
Change description of cohort selection panel in Aggregate Feature Imp…
gaugup Oct 13, 2022
18af5ab
Support cohort filtering of string target in rai_insights (#1771)
gaugup Oct 18, 2022
c23be59
Simplify tests in test_cohort_filter.py (#1772)
gaugup Oct 18, 2022
3c9f037
Forecasting Dashboard
t-wangjohn Nov 18, 2022
a575a9d
remove unrelated file
romanlutz Dec 13, 2022
b7d18b6
basic set of changes to get it working again
romanlutz Dec 13, 2022
b14a162
Merge branch 't-wangjohn/forecasting' of https://github.com/microsoft…
romanlutz Dec 13, 2022
fa7c4e3
Merge branch 'main' of https://github.com/microsoft/responsible-ai-to…
romanlutz Dec 13, 2022
59c89f7
forecasting_grains -> time_series_id_column_names
romanlutz Dec 13, 2022
77908d7
fix breaking changes in latest main, add quantile prediction, fix dat…
romanlutz Dec 16, 2022
4e6bd7e
fix UI components (including table and what-if creation, chart still …
romanlutz Dec 22, 2022
c193acc
get dashboard to work with multiple time series and switching back an…
romanlutz Dec 30, 2022
1f8f656
remove data explorer and model overview, create and edit cohort funct…
romanlutz Dec 30, 2022
c505871
lintfix
romanlutz Dec 31, 2022
98b0602
Merge branch 'main' of https://github.com/microsoft/responsible-ai-to…
romanlutz Jan 3, 2023
010b1e4
localizzation
romanlutz Jan 3, 2023
1b6cb73
Merge branch 'main' of https://github.com/microsoft/responsible-ai-to…
romanlutz Jan 4, 2023
19470a1
remove console.log
romanlutz Jan 4, 2023
f243a9c
remove question
romanlutz Jan 4, 2023
7a88711
remove unrelated changes, fix package.json
romanlutz Jan 4, 2023
d9820f7
string fixes and package.json adjustment
romanlutz Jan 4, 2023
fd3e796
localization
romanlutz Jan 4, 2023
e33def0
localization
romanlutz Jan 4, 2023
7211b97
more localization and removing of unrelated files
romanlutz Jan 4, 2023
0adeada
remove unrelated files
romanlutz Jan 4, 2023
031c62e
remove unrelated file changes
romanlutz Jan 4, 2023
ea3b991
remove is_forecasting_true_y
romanlutz Jan 4, 2023
fce894a
lintfix
romanlutz Jan 4, 2023
050f50b
Merge branch 'main' into romanlutz/forecasting_ui
romanlutz Jan 5, 2023
6c1d32a
add dropdown label
romanlutz Jan 5, 2023
3070799
Merge branch 'romanlutz/forecasting_ui' of https://github.com/microso…
romanlutz Jan 5, 2023
274d557
Merge branch 'main' of https://github.com/microsoft/responsible-ai-to…
romanlutz Jan 6, 2023
bc621f2
Merge branch 'main' of https://github.com/microsoft/responsible-ai-to…
romanlutz Jan 7, 2023
6ba1089
Merge branch 'main' of https://github.com/microsoft/responsible-ai-to…
romanlutz Jan 9, 2023
777522b
Merge branch 'main' of https://github.com/microsoft/responsible-ai-to…
romanlutz Jan 12, 2023
0dda1dc
cache transformation predictions
romanlutz Jan 12, 2023
64e250d
lintfix
romanlutz Jan 12, 2023
75c4e05
remove isUndefinedOrEmpty
romanlutz Jan 12, 2023
0698791
remove isUndefinedOrEmpty
romanlutz Jan 12, 2023
81eebd5
rename isAllDataCohort file as requested
romanlutz Jan 12, 2023
edea378
lintfix
romanlutz Jan 12, 2023
d357f1f
Merge branch 'main' into romanlutz/forecasting_ui
romanlutz Jan 12, 2023
7c5ea44
remove unused componentDidMount
romanlutz Jan 13, 2023
0e62031
Merge branch 'main' of https://github.com/microsoft/responsible-ai-to…
romanlutz Jan 13, 2023
334e800
Merge branch 'main' into romanlutz/forecasting_ui
romanlutz Jan 13, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion libs/core-ui/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
// Licensed under the MIT License.

export * from "./lib/cohortKey";
export * from "./lib/Cohort/isAllDataCohort";
export * from "./lib/Cohort/allDataCohortUtils";
export * from "./lib/Cohort/Cohort";
export * from "./lib/Cohort/CohortList/CohortList";
export * from "./lib/Cohort/Constants";
Expand Down
2 changes: 1 addition & 1 deletion libs/core-ui/src/lib/Cohort/CohortInfo/CohortInfo.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@ import { localization } from "@responsible-ai/localization";
import React from "react";

import { getCohortFilterCount } from "../../util/getCohortFilterCount";
import { isAllDataErrorCohort } from "../allDataCohortUtils";
import { ErrorCohortStats } from "../CohortStats";
import { ErrorCohort } from "../ErrorCohort";
import { isAllDataErrorCohort } from "../isAllDataCohort";
import { PredictionPath } from "../PredictionPath/PredictionPath";

import { cohortInfoStyles } from "./CohortInfo.styles";
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import {
IModelAssessmentContext,
ModelAssessmentContext
} from "../../Context/ModelAssessmentContext";
import { isAllDataErrorCohort } from "../isAllDataCohort";
import { isAllDataErrorCohort } from "../allDataCohortUtils";

export interface ICohortInfoSectionProps {
toggleShiftCohortVisibility: () => void;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,15 +14,20 @@ import { SeriesOptionsType } from "highcharts";
import React from "react";

import { forecastingDashboardStyles } from "../ForecastingDashboard.styles";
import { Transformation } from "../Interfaces/Transformation";

import { getForecastPrediction } from "./getForecastPrediction";

export class IForecastComparisonProps {}
export interface IForecastComparisonProps {
transformations: Map<string, Transformation>;
}

export interface IForecastComparisonState {
timeSeriesId?: number;
baselinePrediction?: Array<[number, number]>;
trueY?: Array<[number, number]>;
transformationPredictions: Map<string, Array<[number, number]>>;
selectedTransformations: Set<string>;
}

const stackTokens = {
Expand All @@ -39,7 +44,11 @@ export class ForecastComparison extends React.Component<

public constructor(props: IForecastComparisonProps) {
super(props);
this.state = {};
this.state = {
baselinePrediction: undefined,
selectedTransformations: new Set<string>(),
transformationPredictions: new Map<string, Array<[number, number]>>()
};
}

public async componentDidMount(): Promise<void> {
Expand All @@ -58,11 +67,36 @@ export class ForecastComparison extends React.Component<
if (currentlySelectedTimeSeriesId !== this.state.timeSeriesId) {
const trueY = this.getTrueY();
const baselinePrediction = await this.getBaselineForecastPrediction();
const selectedTransformationsAndPredictions =
await this.getSelectedForecastPredictions(
[...this.props.transformations.keys()],
true
);
this.setState({
baselinePrediction,
selectedTransformations:
selectedTransformationsAndPredictions.selectedTransformations,
timeSeriesId: currentlySelectedTimeSeriesId,
transformationPredictions:
selectedTransformationsAndPredictions.transformationPredictions,
trueY
});
return;
}

// Check if any new transformations were added.
// If so, add their corresponding predictions to this.state.transformationPredictions.
// If we add deletion for transformations we will need to check for transformations
// that have been removed and delete their corresponding predictions, too.
const currentTransformations = [...this.props.transformations.keys()];
const prevTransformations = new Set(
this.state.transformationPredictions.keys()
);
const newlyAddedTransformations = currentTransformations.filter(
(t) => !prevTransformations.has(t)
);
if (newlyAddedTransformations.length > 0) {
this.addSelectedForecastPredictions(newlyAddedTransformations);
}
}

Expand All @@ -89,6 +123,17 @@ export class ForecastComparison extends React.Component<
type: "spline"
} as SeriesOptionsType);
}
this.state.selectedTransformations.forEach((transformationName) => {
const transformationPredictions =
this.state.transformationPredictions.get(transformationName);
if (transformationPredictions) {
seriesData.push({
data: transformationPredictions,
name: transformationName,
type: "spline"
} as SeriesOptionsType);
}
});

return (
<Stack tokens={stackTokens}>
Expand Down Expand Up @@ -155,6 +200,72 @@ export class ForecastComparison extends React.Component<
return undefined;
};

private getSelectedForecastPredictions = async (
newTransformationNames: string[],
ignoreExisting?: boolean
): Promise<{
transformationPredictions: Map<string, Array<[number, number]>>;
selectedTransformations: Set<string>;
}> => {
const newTransformationPredictions = await Promise.all(
newTransformationNames.map(async (newTransformationName) => {
if (this.context.requestForecast === undefined) {
return;
}
const newTransformation = this.props.transformations.get(
newTransformationName
);
if (newTransformation === undefined) {
return;
}

const pred = await getForecastPrediction(
newTransformation.cohort.cohort,
this.context.jointDataset,
this.context.requestForecast,
newTransformation
);
if (pred && this.context.dataset.index) {
return orderByTime(pred, this.getIndices(this.context.dataset.index));
}
return undefined;
})
);

const newMap = ignoreExisting
? new Map<string, Array<[number, number]>>()
: new Map(this.state.transformationPredictions);
const newSet = ignoreExisting
? new Set<string>()
: new Set(this.state.selectedTransformations);
newTransformationNames.forEach((newTransformationName, index) => {
const newPredictions = newTransformationPredictions[index];
if (newPredictions !== undefined) {
newMap.set(newTransformationName, newPredictions);
newSet.add(newTransformationName);
}
});

return {
selectedTransformations: newSet,
transformationPredictions: newMap
};
};

private addSelectedForecastPredictions = async (
newTransformationNames: string[]
): Promise<void> => {
const selectedTransformationsAndPredictions =
await this.getSelectedForecastPredictions(newTransformationNames);

this.setState({
selectedTransformations:
selectedTransformationsAndPredictions.selectedTransformations,
transformationPredictions:
selectedTransformationsAndPredictions.transformationPredictions
});
};

private readonly getTrueY = (): Array<[number, number]> | undefined => {
if (this.context.dataset.index) {
return orderByTime(
Expand Down
Loading