Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Act and Edit are no-ops on inactive maps #155

Merged
merged 4 commits into from
May 24, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions binder/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -42,3 +42,4 @@ RUN pip install --no-cache-dir nbstripout \
USER ${NB_USER}

ENTRYPOINT ["htmap/binder/entrypoint.sh"]
CMD ["jupyter", "lab"]
12 changes: 6 additions & 6 deletions binder/condor_config.local
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,12 @@ EXECUTE=$(LOCAL_DIR)/execute
CRED_STORE_DIR=$(LOCAL_DIR)/cred_dir

# Tuning so jobs start quickly
SCHEDD_INTERVAL=5
NEGOTIATOR_INTERVAL=2
NEGOTIATOR_CYCLE_DELAY=5
STARTER_UPDATE_INTERVAL=5
SHADOW_QUEUE_UPDATE_INTERVAL=10
UPDATE_INTERVAL=5
SCHEDD_INTERVAL=1
NEGOTIATOR_INTERVAL=1
NEGOTIATOR_CYCLE_DELAY=1
STARTER_UPDATE_INTERVAL=1
SHADOW_QUEUE_UPDATE_INTERVAL=1
UPDATE_INTERVAL=1
RUNBENCHMARKS=0

# Don't use all the machine resources
Expand Down
8 changes: 8 additions & 0 deletions binder/edit.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/usr/bin/env bash

set -e

CONTAINER_TAG=htmap-edit-tutorials

docker build -t ${CONTAINER_TAG} --file binder/Dockerfile .
docker run -it --rm -p 8888:8888 --mount type=bind,source="$(pwd)"/docs/source/tutorials,target=/home/jovyan/tutorials ${CONTAINER_TAG}
File renamed without changes.
71 changes: 52 additions & 19 deletions docs/source/tutorials/advanced-mapping.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@
"name": "stdout",
"output_type": "stream",
"text": [
"created map light-red-bird with 3 components\n",
"created map hollow-bland-zebra with 3 components\n",
"[1, 4, 27]\n"
]
}
Expand Down Expand Up @@ -93,7 +93,7 @@
"name": "stdout",
"output_type": "stream",
"text": [
"created map harsh-bland-beak with 3 components\n",
"created map dry-puny-year with 3 components\n",
"[1, 4, 27]\n"
]
}
Expand Down Expand Up @@ -122,7 +122,7 @@
"source": [
"## Mapped Functions\n",
"\n",
"If you're tired of re-typing the name of your function all the time, create a [htmap.MappedFunction](../api.rst#htmap.MappedFunction) using the [htmap.mapped](../api.rst#htmap.mapped) decorator:"
"If you're tired of typing `htmap.map` all the time, create a [htmap.MappedFunction](../api.rst#htmap.MappedFunction) using the [htmap.mapped](../api.rst#htmap.mapped) decorator:"
]
},
{
Expand All @@ -134,7 +134,7 @@
"name": "stdout",
"output_type": "stream",
"text": [
"<MappedFunction(func = <function double at 0x7f52c19b3378>, map_options = {})>\n"
"<MappedFunction(func = <function double at 0x7fddec16a7b8>, map_options = {})>\n"
]
}
],
Expand All @@ -157,14 +157,15 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"created map light-proud-apple with 10 components\n"
"created map dark-dizzy-apple with 10 components\n",
"[0, 2, 4, 6, 8, 10, 12, 14, 16, 18]\n"
]
}
],
Expand All @@ -184,17 +185,17 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
"@htmap.mapped(\n",
" map_options = htmap.MapOptions(\n",
" request_memory = '1GB',\n",
" request_disk = '10GB',\n",
" request_memory = '200MB',\n",
" request_disk = '1GB',\n",
" )\n",
")\n",
"def big_array(_):\n",
"def big_list(_):\n",
" big = list(range(1_000_000)) # imagine this is way bigger...\n",
" return big"
]
Expand Down Expand Up @@ -228,7 +229,7 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -245,12 +246,12 @@
" print('mmm, yummy fish')\n",
" \n",
" def fly(self):\n",
" raise TypeError(\"penguins can't fly\")"
" raise TypeError(\"penguins can't fly!\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -263,12 +264,34 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 9,
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"penguin-stats: 0%| | 0/3 [00:00<?, ?component/s]"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"created map penguin-stats with 3 components\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"penguin-stats: 100%|##########| 3/3 [00:11<00:00, 1.31s/component]\n"
]
}
],
"source": [
"map = htmap.map(\n",
" lambda p: p.analyze(),\n",
" lambda p: p.analyze(), # an anonmyous function; see https://docs.python.org/3/tutorial/controlflow.html#lambda-expressions\n",
" penguins,\n",
" tag = 'penguin-stats',\n",
")\n",
Expand All @@ -278,9 +301,19 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 10,
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Gwendolin is 73 inches tall and weighs 51 pounds\n",
"Gweniffer is 59 inches tall and weighs 43 pounds\n",
"Gary is 64 inches tall and weighs 52 pounds\n"
]
}
],
"source": [
"for stats in map:\n",
" print(stats)"
Expand All @@ -290,7 +323,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Specialized data structures like `numpy` arrays and `pandas` dataframes can also be used as function arguments."
"Specialized data structures like `numpy` arrays and `pandas` dataframes can also be used as function arguments. When in doubt, just try it!"
]
},
{
Expand Down
Loading