Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: profiling in EDA tool #954

Merged
merged 47 commits into from
Apr 6, 2024
Merged
Show file tree
Hide file tree
Changes from 39 commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
7250d2b
fix: eda -> runner comms to own API file & no more undefined eda vars
SmiteDeluxe Feb 29, 2024
217cc3f
fix: undefined state on reveal panel
SmiteDeluxe Feb 29, 2024
bc1ac24
fix: waitForUpdateHtmlDone now not part of constructCurrentState anymore
SmiteDeluxe Feb 29, 2024
5afb4df
fix: small things & feat: first runnerApi methods that append pipeline
SmiteDeluxe Feb 29, 2024
00a491d
fix: properly show "0" values in table
SmiteDeluxe Mar 1, 2024
4709cc3
feat: get and set basic profiling for new table
SmiteDeluxe Mar 1, 2024
7a0481b
Merge remote-tracking branch 'origin/main' into 929-eda-append-code-t…
SmiteDeluxe Mar 1, 2024
926d79c
fix: col/row deselection if click row/col again and only one
SmiteDeluxe Mar 1, 2024
0b368f7
fix: profiling minimal better styling
SmiteDeluxe Mar 1, 2024
f220bf5
feat: optionally pass table to getProfiling for performance
SmiteDeluxe Mar 1, 2024
e953262
feat: more profiling info & profiling styling
SmiteDeluxe Mar 1, 2024
cebbda6
fix: ProfilingInfo own comp & better profiling height calc
SmiteDeluxe Mar 5, 2024
2789859
fix: profiling fetch also if state existing but no profiling info
SmiteDeluxe Mar 5, 2024
2cb1da6
fix: icon colors & profiling height bug & warn icon if missing vals
SmiteDeluxe Mar 5, 2024
6ee653e
feat: filter and sort icons in col headers
SmiteDeluxe Mar 5, 2024
d9ba845
feat: filter context menu start & fix: row/col select clear
SmiteDeluxe Mar 6, 2024
0d476f8
Merge remote-tracking branch 'origin/main' into 929-eda-append-code-t…
SmiteDeluxe Mar 7, 2024
6237bfc
fix: multiple pipeline support & refactoring
SmiteDeluxe Mar 7, 2024
8350e3c
feat: CODEGEN_PREFIX in front of generated placeholders
SmiteDeluxe Mar 7, 2024
5a532c9
fix: rename current warn to error & no color in state, decided in svelte
SmiteDeluxe Mar 7, 2024
18a24e0
fix: updateTableSpace column names trimmed & minor things
SmiteDeluxe Mar 8, 2024
95734a4
fix: webview reload and profiling semantic
SmiteDeluxe Mar 29, 2024
44b3850
fix: no manual updateTableSpace & colWidths svelte store & head has bg
SmiteDeluxe Mar 29, 2024
8126561
fix: change state profiling semantics
SmiteDeluxe Mar 29, 2024
608cce2
fix: placeholder generation now incr number
SmiteDeluxe Mar 29, 2024
a4fd31c
feat: profiling histograms
SmiteDeluxe Mar 29, 2024
d27a4de
Merge remote-tracking branch 'origin/main' into 929-eda-append-code-t…
SmiteDeluxe Mar 30, 2024
7306942
refactor: some comments and code, more readable
SmiteDeluxe Mar 30, 2024
6e3ab4f
fix: image in top of profiling
SmiteDeluxe Mar 30, 2024
5e951a6
Merge remote-tracking branch 'origin/main' into 929-eda-append-code-t…
SmiteDeluxe Mar 30, 2024
6c6259b
refactor: comments for readability and future
SmiteDeluxe Mar 30, 2024
e8e792a
fix: mega-linter issues
SmiteDeluxe Mar 30, 2024
ff8e14e
fix: small annoying megalinter fix
SmiteDeluxe Mar 30, 2024
0bac77d
fix: pr comments
SmiteDeluxe Apr 2, 2024
4b27452
Merge remote-tracking branch 'origin/main' into 929-eda-append-code-t…
SmiteDeluxe Apr 2, 2024
4d37f5f
fix: megalinter
SmiteDeluxe Apr 2, 2024
131aab7
Merge remote-tracking branch 'origin/main' into 929-eda-append-code-t…
SmiteDeluxe Apr 3, 2024
5784f35
Merge remote-tracking branch 'origin/main' into 929-eda-append-code-t…
SmiteDeluxe Apr 3, 2024
cf64b65
Merge branch 'main' into 929-eda-append-code-to-pipelines
lars-reimann Apr 3, 2024
7f5b64b
Merge branch 'main' into 929-eda-append-code-to-pipelines
WinPlay02 Apr 3, 2024
7427d6d
fix: pr comments & pipeline execute reject in runnerApi
SmiteDeluxe Apr 4, 2024
43ff883
style: apply automated linter fixes
megalinter-bot Apr 4, 2024
1d98097
Merge branch 'main' into 929-eda-append-code-to-pipelines
WinPlay02 Apr 4, 2024
98c113d
Merge remote-tracking branch 'origin/main' into 929-eda-append-code-t…
SmiteDeluxe Apr 5, 2024
12e8fd6
Merge remote-tracking branch 'origin/main' into 929-eda-append-code-t…
SmiteDeluxe Apr 5, 2024
2b8cbf5
Merge branch 'main' into 929-eda-append-code-to-pipelines
lars-reimann Apr 6, 2024
697c66f
Merge branch 'main' into 929-eda-append-code-to-pipelines
lars-reimann Apr 6, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions packages/safe-ds-eda/src/App.svelte
Original file line number Diff line number Diff line change
Expand Up @@ -37,9 +37,9 @@
</script>

<main>
<div class="sidebarWrapper" style="width: {sidebarWidth}px;" class:white-bg={sidebarWidth < 100}>
<div class="sidebarWrapper" style:width="{sidebarWidth}px" class:white-bg={sidebarWidth < 100}>
<Sidebar width={sidebarWidth} />
<div class="resizer" on:mousedown={handleDrag}></div>
<button class="resizer" on:mousedown={handleDrag}></button>
</div>
<div class="tableWrapper">
<TableView {sidebarWidth} />
Expand Down Expand Up @@ -68,7 +68,6 @@

.tableWrapper {
flex: 1;
overflow: scroll;
}

.resizer {
Expand Down
739 changes: 483 additions & 256 deletions packages/safe-ds-eda/src/components/TableView.svelte

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
<script lang="ts">
import type { PossibleColumnFilter } from '../../../types/state';

export let possibleFilters: PossibleColumnFilter[];

const handleSpecificValueFilterChange = (_event: Event) => {
// TODO: Implement
};
</script>

<div class="wrapper">
{#each possibleFilters as filter}
{#if filter.type === 'specificValue'}
<div class="filterRow contextItem">
<span class="filterName">Filter by value</span>
<select on:change={handleSpecificValueFilterChange}>
{#each filter.values as value}
<option {value}>{value}</option>
{/each}
</select>
</div>
{/if}
{/each}
</div>

<style>
.wrapper {
display: flex;
flex-direction: column;
gap: 10px;
}

.filterRow {
display: flex;
flex-direction: row;
gap: 5px;
}

.filterName {
color: var(--font-light);
}
</style>
100 changes: 100 additions & 0 deletions packages/safe-ds-eda/src/components/profiling/ProfilingInfo.svelte
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
<script lang="ts">
import type { Profiling } from '../../../types/state';

export let profiling: Profiling;
export let imageWidth: number = 200;
</script>

<div class="wrapper">
<div class="profilingItemsTop">
<div class="profilingItem">
<span class="profilingItemKey {profiling.validRatio.interpretation}">{profiling.validRatio.name}:</span>
<span class={profiling.validRatio.interpretation}>{profiling.validRatio.value}</span>
</div>
<div class="profilingItem">
<span class="profilingItemKey {profiling.missingRatio.interpretation}">{profiling.missingRatio.name}:</span>
<span class={profiling.missingRatio.interpretation}>{profiling.missingRatio.value}</span>
</div>
</div>
<div class="profilingItemsBottom">
{#each profiling.other as profilingItem}
{#if profilingItem.type === 'text'}
<div class="profilingItem">
<span class={profilingItem.interpretation}>{profilingItem.value}</span>
</div>
{:else if profilingItem.type === 'numerical'}
<div class="profilingItem">
<span class="profilingItemKey {profilingItem.interpretation}">{profilingItem.name}:</span>
<span class={profilingItem.interpretation}>{profilingItem.value}</span>
</div>
{:else if profilingItem.type === 'image'}
<div class="profilingItem">
<img
style:width="{imageWidth}px"
class="profilingImage"
src={'data:image/' + profilingItem.value.format + ';base64,' + profilingItem.value.bytes}
alt="profiling plot"
/>
</div>
{/if}
{/each}
</div>
</div>

<style>
.wrapper {
height: 100%;
display: flex;
flex-direction: column;
justify-content: space-between;
}

.profilingItem {
display: flex;
justify-content: space-between;
margin-bottom: 1px;
}

.profilingItemKey {
margin-right: 10px;
}

.profilingItemsTop {
margin-bottom: 20px;
width: 100%;
}

.profilingItemsBottom {
width: 100%;
}

.profilingImage {
height: 150px; /* default height, profiling height calculation works off this value */
object-fit: cover;
object-position: left;
}

.good {
color: var(--primary-color);
}

.error {
color: var(--error-color);
}

.warn {
color: var(--warn-color);
}

.important {
color: var(--font-dark);
}

.default {
color: var(--font-light);
}

.category {
color: var(--font-light);
}
</style>
2 changes: 1 addition & 1 deletion packages/safe-ds-eda/src/icons/BarPlot.svelte
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<svg width="12" height="9" viewBox="0 0 12 9" fill="none" xmlns="http://www.w3.org/2000/svg">
<path
d="M7.57893 5.82995H8.45343C8.59918 5.82995 8.74492 5.6842 8.74492 5.53845V2.47773C8.74492 2.33198 8.59918 2.18623 8.45343 2.18623H7.57893C7.43319 2.18623 7.28744 2.33198 7.28744 2.47773V5.53845C7.28744 5.6842 7.43319 5.82995 7.57893 5.82995ZM9.76516 5.82995H10.6397C10.7854 5.82995 10.9312 5.6842 10.9312 5.53845V0.291498C10.9312 0.145749 10.7854 0 10.6397 0H9.76516C9.61942 0 9.47367 0.145749 9.47367 0.291498V5.53845C9.47367 5.6842 9.61942 5.82995 9.76516 5.82995ZM3.20647 5.82995H4.08096C4.22671 5.82995 4.37246 5.6842 4.37246 5.53845V3.93522C4.37246 3.78947 4.22671 3.64372 4.08096 3.64372H3.20647C3.06072 3.64372 2.91497 3.78947 2.91497 3.93522V5.53845C2.91497 5.6842 3.06072 5.82995 3.20647 5.82995ZM5.3927 5.82995H6.2672C6.41294 5.82995 6.55869 5.6842 6.55869 5.53845V1.02024C6.55869 0.874492 6.41294 0.728744 6.2672 0.728744H5.3927C5.24695 0.728744 5.10121 0.874492 5.10121 1.02024V5.53845C5.10121 5.6842 5.24695 5.82995 5.3927 5.82995ZM11.2955 7.28744H1.45749V0.364372C1.45749 0.163056 1.29443 0 1.09312 0H0.364372C0.163056 0 0 0.163056 0 0.364372V8.01618C0 8.41858 0.326341 8.74492 0.728744 8.74492H11.2955C11.4968 8.74492 11.6599 8.58187 11.6599 8.38055V7.65181C11.6599 7.45049 11.4968 7.28744 11.2955 7.28744Z"
fill="#036ED1"
fill="var(--primary-color)"
/>
</svg>
6 changes: 5 additions & 1 deletion packages/safe-ds-eda/src/icons/Caret.svelte
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
<script lang="ts">
export let color = 'var(--font-light)';
</script>

<svg width="13" height="11" viewBox="0 0 5 4" fill="none" xmlns="http://www.w3.org/2000/svg">
<path d="M2.5 0.5L4.66506 3.5H0.334937L2.5 0.5Z" fill="#636363" />
<path d="M2.5 0.5L4.66506 3.5H0.334937L2.5 0.5Z" fill={color} />
</svg>
6 changes: 6 additions & 0 deletions packages/safe-ds-eda/src/icons/Error.svelte
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512"
><!--!Font Awesome Free 6.5.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free Copyright 2024 Fonticons, Inc.--><path
d="M256 512A256 256 0 1 0 256 0a256 256 0 1 0 0 512zm0-384c13.3 0 24 10.7 24 24V264c0 13.3-10.7 24-24 24s-24-10.7-24-24V152c0-13.3 10.7-24 24-24zM224 352a32 32 0 1 1 64 0 32 32 0 1 1 -64 0z"
fill="var(--error-color)"
/></svg
>
10 changes: 10 additions & 0 deletions packages/safe-ds-eda/src/icons/Filter.svelte
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
<script lang="ts">
export let color = 'var(--transparent)';
</script>

<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512"
><!--!Font Awesome Free 6.5.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free Copyright 2024 Fonticons, Inc.--><path
d="M3.9 54.9C10.5 40.9 24.5 32 40 32H472c15.5 0 29.5 8.9 36.1 22.9s4.6 30.5-5.2 42.5L320 320.9V448c0 12.1-6.8 23.2-17.7 28.6s-23.8 4.3-33.5-3l-64-48c-8.1-6-12.8-15.5-12.8-25.6V320.9L9 97.3C-.7 85.4-2.8 68.8 3.9 54.9z"
fill={color}
/></svg
>
2 changes: 1 addition & 1 deletion packages/safe-ds-eda/src/icons/LinePlot.svelte
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<svg width="19" height="15" viewBox="0 0 12 9" fill="none" xmlns="http://www.w3.org/2000/svg">
<path
d="M11.2955 7.28744H1.45749V0.364372C1.45749 0.163056 1.29443 0 1.09312 0H0.364372C0.163056 0 0 0.163056 0 0.364372V8.01618C0 8.41858 0.326341 8.74492 0.728744 8.74492H11.2955C11.4968 8.74492 11.6599 8.58187 11.6599 8.38055V7.65181C11.6599 7.45049 11.4968 7.28744 11.2955 7.28744ZM10.5668 0.728744H7.87817C7.39128 0.728744 7.14738 1.31743 7.49171 1.66176L8.22957 2.39962L6.55869 4.07072L4.88782 2.39984C4.60316 2.11518 4.14177 2.11518 3.85733 2.39984L2.29304 3.96414C2.1507 4.10647 2.1507 4.33716 2.29304 4.4795L2.80817 4.99463C2.9505 5.13696 3.18119 5.13696 3.32353 4.99463L4.37246 3.94546L6.04333 5.61634C6.328 5.901 6.78939 5.901 7.07382 5.61634L9.26005 3.4301L9.99791 4.16796C10.3422 4.51229 10.9309 4.26839 10.9309 3.7815V1.09312C10.9312 0.8918 10.7681 0.728744 10.5668 0.728744Z"
fill="#036ED1"
fill="var(--primary-color)"
/>
</svg>
2 changes: 1 addition & 1 deletion packages/safe-ds-eda/src/icons/Table.svelte
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<svg width="20" height="18" viewBox="0 0 13 12" fill="none" xmlns="http://www.w3.org/2000/svg">
<path
d="M11.5038 0.963623H2.03013C1.42641 0.963623 0.937012 1.45302 0.937012 2.05674V10.0729C0.937012 10.6766 1.42641 11.166 2.03013 11.166H11.5038C12.1075 11.166 12.5969 10.6766 12.5969 10.0729V2.05674C12.5969 1.45302 12.1075 0.963623 11.5038 0.963623ZM6.03822 9.70855H2.3945V7.52232H6.03822V9.70855ZM6.03822 6.06483H2.3945V3.8786H6.03822V6.06483ZM11.1394 9.70855H7.4957V7.52232H11.1394V9.70855ZM11.1394 6.06483H7.4957V3.8786H11.1394V6.06483Z"
fill="#036ED1"
fill="var(--primary-color)"
/>
</svg>
2 changes: 1 addition & 1 deletion packages/safe-ds-eda/src/icons/Undo.svelte
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<svg width="18" height="18" viewBox="0 0 13 13" fill="none" xmlns="http://www.w3.org/2000/svg">
<path
d="M5.84604 6.15008H1.21445C1.06123 6.15008 0.937012 6.02586 0.937012 5.87265V1.24106C0.937012 1.08784 1.06123 0.963623 1.21445 0.963623H2.32418C2.47739 0.963623 2.60161 1.08784 2.60161 1.24106V3.04696C3.65986 1.87173 5.197 1.13633 6.90588 1.14874C10.0711 1.17172 12.6036 3.7294 12.5969 6.89467C12.5902 10.0555 10.0257 12.6158 6.86329 12.6158C5.38159 12.6158 4.03125 12.0537 3.01367 11.1312C2.89579 11.0244 2.89035 10.841 3.00287 10.7285L3.78817 9.94318C3.89161 9.83974 4.05779 9.83412 4.16736 9.93104C4.88489 10.566 5.82863 10.9512 6.86329 10.9512C9.11208 10.9512 10.9323 9.13136 10.9323 6.88221C10.9323 4.63344 9.11245 2.81318 6.86329 2.81318C5.5109 2.81318 4.31368 3.47153 3.57399 4.48548H5.84604C5.99925 4.48548 6.12347 4.6097 6.12347 4.76291V5.87265C6.12347 6.02586 5.99925 6.15008 5.84604 6.15008Z"
fill="#036ED1"
fill="var(--primary-color)"
/>
</svg>
30 changes: 28 additions & 2 deletions packages/safe-ds-eda/src/webviewState.ts
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
import type { FromExtensionMessage } from '../types/messaging';
import type { State } from '../types/state';
import * as extensionApi from './apis/extensionApi';
import { writable } from 'svelte/store';
import { get, writable } from 'svelte/store';

let currentTabIndex = writable<number>(0);

let preventClicks = writable<boolean>(false);

// Define the stores, current state to default in case the extension never calls setWebviewState( Shouldn't happen)
let currentState = writable<State>({ tableIdentifier: window.tableIdentifier, history: [], defaultState: true });
let currentState = writable<State>({ tableIdentifier: undefined, history: [], defaultState: true });

// Set Global states whenever updatedAllStates changes
currentState.subscribe(($currentState) => {
Expand All @@ -26,6 +26,32 @@ window.addEventListener('message', (event) => {
// This should be fired immediately whenever the panel is created or made visible again
currentState.set(message.value);
break;
case 'setProfiling':
if (get(currentState) && get(currentState).table) {
currentState.update((state) => {
return {
...state,
table: {
...state.table!,
columns: state.table!.columns.map((column) => {
lars-reimann marked this conversation as resolved.
Show resolved Hide resolved
const profiling = message.value.find((p) => p.columnName === column[1].name);
if (profiling) {
return [
column[0],
lars-reimann marked this conversation as resolved.
Show resolved Hide resolved
{
...column[1],
profiling: profiling.profiling,
},
];
} else {
return column;
}
}),
},
};
});
}
break;
}
});

Expand Down
1 change: 0 additions & 1 deletion packages/safe-ds-eda/types/globals.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,5 @@ declare global {
injVscode: {
postMessage: (message: ToExtensionMessage) => void;
};
tableIdentifier: string;
}
}
7 changes: 6 additions & 1 deletion packages/safe-ds-eda/types/messaging.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -46,4 +46,9 @@ interface FromExtensionSetStateMessage extends FromExtensionCommandMessage {
value: defaultTypes.State;
}

export type FromExtensionMessage = FromExtensionSetStateMessage;
interface FromExtensionSetProfilingMessage extends FromExtensionCommandMessage {
command: 'setProfiling';
value: { columnName: string; profiling: defaultTypes.Profiling }[];
}

export type FromExtensionMessage = FromExtensionSetStateMessage | FromExtensionSetProfilingMessage;
Loading