WIP: notebooks, plus improved merge_delete_downstream

CBroz1 · Aug 9, 2023 · 456cc86 · 456cc86
1 parent 7a46b11
commit 456cc86
Show file tree

Hide file tree

Showing 9 changed files with 549 additions and 239 deletions.
diff --git a/notebooks/01_Insert_Data.ipynb b/notebooks/01_Insert_Data.ipynb
@@ -667,9 +667,9 @@
     "By adding diagrams together, of adding and subtracting levels, we can visualize\n",
     "key parts of Spyglass.\n",
     "\n",
-    "_Note:_ Notice the *Selection* tables. This is a design pattern that selects a \n",
+    "_Note:_ Notice the _Selection_ tables. This is a design pattern that selects a\n",
     "subset of upstream items for further processing. In some cases, these also pair\n",
-    "the selected data with processing parameters."
+    "the selected data with processing parameters.\n"
    ]
   },
   {
@@ -2147,57 +2147,16 @@
    "metadata": {},
    "source": [
     "By default, DataJoint is cautious about deletes and will prompt before deleting.\n",
-    "To delete, respond `yes` in the prompt.\n"
+    "To delete, uncomment the cell below and respond `yes` in the prompt.\n"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 50,
+   "execution_count": null,
    "metadata": {},
-   "outputs": [
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "[2023-07-18 18:51:35,382][INFO]: Deleting 4 rows from `common_behav`.`_raw_position`\n",
-      "INFO:datajoint:Deleting 4 rows from `common_behav`.`_raw_position`\n",
-      "[2023-07-18 18:51:35,388][INFO]: Deleting 4 rows from `common_behav`.`position_source`\n",
-      "INFO:datajoint:Deleting 4 rows from `common_behav`.`position_source`\n",
-      "[2023-07-18 18:51:35,393][INFO]: Deleting 7 rows from `common_dio`.`_d_i_o_events`\n",
-      "INFO:datajoint:Deleting 7 rows from `common_dio`.`_d_i_o_events`\n",
-      "[2023-07-18 18:51:35,406][INFO]: Deleting 128 rows from `common_ephys`.`_electrode`\n",
-      "INFO:datajoint:Deleting 128 rows from `common_ephys`.`_electrode`\n",
-      "[2023-07-18 18:51:35,408][INFO]: Deleting 1 rows from `common_ephys`.`_electrode_group`\n",
-      "INFO:datajoint:Deleting 1 rows from `common_ephys`.`_electrode_group`\n",
-      "[2023-07-18 18:51:35,412][INFO]: Deleting 1 rows from `common_ephys`.`_raw`\n",
-      "INFO:datajoint:Deleting 1 rows from `common_ephys`.`_raw`\n",
-      "[2023-07-18 18:51:35,415][INFO]: Deleting 1 rows from `common_ephys`.`_sample_count`\n",
-      "INFO:datajoint:Deleting 1 rows from `common_ephys`.`_sample_count`\n",
-      "[2023-07-18 18:51:35,426][INFO]: Deleting 2 rows from `common_task`.`_task_epoch`\n",
-      "INFO:datajoint:Deleting 2 rows from `common_task`.`_task_epoch`\n",
-      "[2023-07-18 18:51:35,434][INFO]: Deleting 7 rows from `common_interval`.`interval_list`\n",
-      "INFO:datajoint:Deleting 7 rows from `common_interval`.`interval_list`\n",
-      "[2023-07-18 18:51:35,441][INFO]: Deleting 1 rows from `common_session`.`_session__data_acquisition_device`\n",
-      "INFO:datajoint:Deleting 1 rows from `common_session`.`_session__data_acquisition_device`\n",
-      "[2023-07-18 18:51:35,442][INFO]: Deleting 1 rows from `common_session`.`_session`\n",
-      "INFO:datajoint:Deleting 1 rows from `common_session`.`_session`\n",
-      "[2023-07-18 18:51:36,986][INFO]: Deletes committed.\n",
-      "INFO:datajoint:Deletes committed.\n"
-     ]
-    },
-    {
-     "data": {
-      "text/plain": [
-       "1"
-      ]
-     },
-     "execution_count": 50,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
+   "outputs": [],
    "source": [
-    "session_entry.delete()"
+    "# session_entry.delete()"
    ]
   },
   {
@@ -2459,13 +2418,21 @@
     }
    ],
    "source": [
-    "# Let's delete the entry\n",
-    "(sgc.Nwbfile & {\"nwb_file_name\": nwb_copy_file_name}).delete()"
+    "# Uncomment to delete\n",
+    "# (sgc.Nwbfile & {\"nwb_file_name\": nwb_copy_file_name}).delete()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Note that the file (ends with `_.nwb`) has not been deleted, even if the entry\n",
+    "was deleted above.\n"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 54,
+   "execution_count": null,
    "metadata": {},
    "outputs": [
     {
@@ -2484,10 +2451,16 @@
     }
    ],
    "source": [
-    "# Note that the file (ends with _.nwb) has not been deleted, even though the entry is\n",
     "!ls $SPYGLASS_BASE_DIR/raw"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We can clean these files with the `cleanup` method\n"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 55,
@@ -2502,7 +2475,6 @@
     }
    ],
    "source": [
-    "# We clean it up\n",
     "sgc.Nwbfile().cleanup(delete_files=True)"
    ]
   },
@@ -2526,7 +2498,6 @@
     }
    ],
    "source": [
-    "# Now the file is gone as well\n",
     "!ls $SPYGLASS_BASE_DIR/raw"
    ]
   },