Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Production Release #203

Merged
merged 38 commits into from
Sep 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
9e07ff5
feat: added download tasl boundaries
Pradip-p Sep 2, 2024
d412759
refractor: changes TaskState schemas to Task schema
Pradip-p Sep 2, 2024
234fbb4
fix: issues reslove on single task area download
Pradip-p Sep 2, 2024
6e01f77
feat: added exception handling on get task geometry
Pradip-p Sep 2, 2024
707ca6c
feat: api download if split_area is false
Pradip-p Sep 2, 2024
e693287
fix: remove commented code
Pradip-p Sep 2, 2024
8bccf67
feat: Automatically approve task locking when requested by project owner
Pradip-p Sep 2, 2024
1c6c174
chore: remove fmtm-splitter package from PDM dependencies
Pradip-p Sep 4, 2024
733e8f7
refactor: remove commented-out code
Pradip-p Sep 4, 2024
7b63837
Fix: Update response to include success message for profile updates
Pradip-p Sep 4, 2024
519af6f
fix: remove print line from user schemas
Pradip-p Sep 4, 2024
9cab6bc
refactor: remove commented-out code
Pradip-p Sep 5, 2024
0d843f2
refactor: email templates name for notification
Pradip-p Sep 5, 2024
f861374
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 5, 2024
9d20eba
Merge branch 'develop' of github.com:hotosm/drone-tm into feat/improv…
Pradip-p Sep 6, 2024
29449d6
Merge pull request #185 from hotosm/feat/download-task-boundaries
nrjadkry Sep 6, 2024
8919e4c
feat: added the dynamic site url for email notifications
Pradip-p Sep 6, 2024
6adf2f7
fix(project-dashboard): project list responsive
suzit-10 Sep 6, 2024
03f9c82
feat: added the default site url for localhost
Pradip-p Sep 6, 2024
a1dceaa
fix: changes site_url to frontend url
Pradip-p Sep 6, 2024
ebea3fd
Merge pull request #192 from hotosm/feat/improve-code-structure
nrjadkry Sep 6, 2024
41a5d2c
feat: updated the directory of dem and map images on s3 bucket
Pradip-p Sep 6, 2024
c531bb5
feat: updated the directory of task images path
Pradip-p Sep 6, 2024
96befe9
Merge pull request #195 from hotosm/refactor/s3-bucket-path
nrjadkry Sep 6, 2024
1c8d776
feat(project-dashboard): visible map by default
suzit-10 Sep 6, 2024
bf630e1
feat(project-dashboard): remove center to nepal and set center to [0,…
suzit-10 Sep 6, 2024
7c9cf99
feat(project-dashboard): increase project centroid pointing circle ra…
suzit-10 Sep 6, 2024
b9112ec
feat(project-dashboard): show popup on project click, the popup conte…
suzit-10 Sep 6, 2024
dafc584
feat(task-description): download waypoint geojson
suzit-10 Sep 6, 2024
9149ee1
Merge branch 'develop' of github.com:hotosm/drone-tm into feat/add-pr…
suzit-10 Sep 6, 2024
7739ffa
feat(user-profile): hide country_code
suzit-10 Sep 8, 2024
ceb1065
fix(user-profile)-#191: issue on user profile update
suzit-10 Sep 8, 2024
5009c99
Merge pull request #196 from hotosm/feat/add-project-image
nrjadkry Sep 9, 2024
fe8b441
fix: dem path when downloading flightplan
nrjadkry Sep 9, 2024
4e26c20
update: waypoints download, altitude added in the waypoints
nrjadkry Sep 9, 2024
838e7aa
Merge branch 'develop' of github.com:hotosm/Drone-TM into fix-downloa…
nrjadkry Sep 9, 2024
606cd20
Update text in the download waypoints geojson
nrjadkry Sep 9, 2024
2d33af2
Merge pull request #199 from hotosm/fix-download-flightplan
nrjadkry Sep 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions src/backend/app/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,7 @@ def assemble_db_connection(cls, v: Optional[str], info: ValidationInfo) -> Any:
)
return pg_url

FRONTEND_URL: str = "http://localhost:3040"
S3_ENDPOINT: str = "http://s3:9000"
S3_ACCESS_KEY: Optional[str] = ""
S3_SECRET_KEY: Optional[str] = ""
Expand Down
2 changes: 0 additions & 2 deletions src/backend/app/drones/drone_logic.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,6 @@
from loguru import logger as log
from fastapi import HTTPException
from psycopg import Connection

# from asyncpg import UniqueViolationError
from typing import List
from app.drones.drone_schemas import DroneOut

Expand Down
2 changes: 1 addition & 1 deletion src/backend/app/drones/drone_routes.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ async def read_drones(
raise HTTPException(status_code=HTTPStatus.NOT_FOUND) from e


@router.post("/create_drone")
@router.post("/create-drone")
async def create_drone(
drone_info: drone_schemas.DroneIn,
db: Annotated[Connection, Depends(database.get_db)],
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,9 @@ <h2>{{ task_status|capitalize }} Task Details</h2>
<p><strong>Description:</strong> {{ description }}</p>
</div>
{% if task_status == 'approved' %}
<a href="https://dronetm-dev.naxa.com.np" class="task-button"
<a
href="{{FRONTEND_URL}}/projects/{{project_id}}/tasks/{{task_id}}"
class="task-button"
>Start Mapping</a
>
{% endif %}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,9 @@ <h2>Mapping Task Details</h2>
<p><strong>Project:</strong>{{project_name}}</p>
<p><strong>Description:</strong> {{description}}</p>
</div>
<a href="dronetm.naxa.com.np" class="task-button"
<a
href="{{FRONTEND_URL}}/projects/{{project_id}}/tasks/{{task_id}}"
class="task-button"
>Visit Drone Tasking Manager</a
>
</div>
Expand Down
16 changes: 5 additions & 11 deletions src/backend/app/projects/project_logic.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@


async def upload_file_to_s3(
project_id: uuid.UUID, file: UploadFile, folder: str, file_extension: str
project_id: uuid.UUID, file: UploadFile, file_name: str
) -> str:
"""
Upload a file (image or DEM) to S3.
Expand All @@ -28,14 +28,8 @@ async def upload_file_to_s3(
Returns:
str: The S3 URL for the uploaded file.
"""
# If the folder is 'images', use 'screenshot.png' as the filename
if folder == "images":
file_name = "screenshot.png"
else:
file_name = f"dem.{file_extension}"

# Define the S3 file path
file_path = f"/{folder}/{project_id}/{file_name}"
file_path = f"/projects/{project_id}/{file_name}"

# Read the file bytes
file_bytes = await file.read()
Expand All @@ -55,7 +49,7 @@ async def upload_file_to_s3(
return file_url


async def update_url(db: Connection, project_id: uuid.UUID, url: str, url_type: str):
async def update_url(db: Connection, project_id: uuid.UUID, url: str):
"""
Update the URL (DEM or image) for a project in the database.

Expand All @@ -70,9 +64,9 @@ async def update_url(db: Connection, project_id: uuid.UUID, url: str, url_type:
"""
async with db.cursor() as cur:
await cur.execute(
f"""
"""
UPDATE projects
SET {url_type} = %(url)s
SET dem_url = %(url)s
WHERE id = %(project_id)s""",
{"url": url, "project_id": project_id},
)
Expand Down
84 changes: 78 additions & 6 deletions src/backend/app/projects/project_routes.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,19 @@
import os
from typing import Annotated
from typing import Annotated, Optional
from uuid import UUID
import geojson
from datetime import timedelta
from fastapi import APIRouter, HTTPException, Depends, UploadFile, File, Form
from fastapi import (
APIRouter,
HTTPException,
Depends,
Path,
Query,
UploadFile,
File,
Form,
Response,
)
from geojson_pydantic import FeatureCollection
from loguru import logger as log
from psycopg import Connection
Expand All @@ -16,13 +27,74 @@
from app.config import settings
from app.users.user_deps import login_required
from app.users.user_schemas import AuthUser
from app.tasks import task_schemas

router = APIRouter(
prefix=f"{settings.API_PREFIX}/projects",
responses={404: {"description": "Not found"}},
)


@router.get("/{project_id}/download-boundaries", tags=["Projects"])
async def download_boundaries(
project_id: Annotated[
UUID,
Path(
description="The project ID in UUID format.",
),
],
db: Annotated[Connection, Depends(database.get_db)],
user_data: Annotated[AuthUser, Depends(login_required)],
task_id: Optional[UUID] = Query(
default=None,
description="The task ID in UUID format. If not provided, all tasks will be downloaded.",
),
split_area: bool = Query(
default=False,
description="Whether to split the area or not. Set to True to download task boundaries, otherwise AOI will be downloaded.",
),
):
"""Downloads the AOI or task boundaries for a project as a GeoJSON file.

Args:
project_id (UUID): The ID of the project in UUID format.
db (Connection): The database connection, provided automatically.
user_data (AuthUser): The authenticated user data, checks if the user has permission.
task_id (Optional[UUID]): The task ID in UUID format. If not provided and split_area is True, all tasks will be downloaded.
split_area (bool): Whether to split the area or not. Set to True to download task boundaries, otherwise AOI will be downloaded.

Returns:
Response: The HTTP response object containing the downloaded file.
"""
try:
out = await task_schemas.Task.get_task_geometry(
db, project_id, task_id, split_area
)

if out is None:
raise HTTPException(status_code=404, detail="Geometry not found.")

filename = (
(f"task_{task_id}.geojson" if task_id else "project_outline.geojson")
if split_area
else "project_aoi.geojson"
)

headers = {
"Content-Disposition": f"attachment; filename={filename}",
"Content-Type": "application/geo+json",
}
return Response(content=out, headers=headers)

except HTTPException as e:
log.error(f"Error during boundaries download: {e.detail}")
raise e

except Exception as e:
log.error(f"Unexpected error during boundaries download: {e}")
raise HTTPException(status_code=500, detail="Internal server error.")


@router.delete("/{project_id}", tags=["Projects"])
async def delete_project_by_id(
project: Annotated[
Expand Down Expand Up @@ -61,16 +133,16 @@ async def create_project(

# Upload DEM and Image to S3
dem_url = (
await project_logic.upload_file_to_s3(project_id, dem, "dem", "tif")
await project_logic.upload_file_to_s3(project_id, dem, "dem.tif")
if dem
else None
)
await project_logic.upload_file_to_s3(
project_id, image, "images", "png"
project_id, image, "map_screenshot.png"
) if image else None

# Update DEM and Image URLs in the database
await project_logic.update_url(db, project_id, dem_url, "dem_url")
await project_logic.update_url(db, project_id, dem_url)

if not project_id:
raise HTTPException(
Expand Down Expand Up @@ -168,7 +240,7 @@ async def generate_presigned_url(
client = s3_client()
urls = []
for image in data.image_name:
image_path = f"publicuploads/{data.project_id}/{data.task_id}/{image}"
image_path = f"projects/{data.project_id}/{data.task_id}/images/{image}"

url = client.get_presigned_url(
"PUT",
Expand Down
2 changes: 1 addition & 1 deletion src/backend/app/projects/project_schemas.py
Original file line number Diff line number Diff line change
Expand Up @@ -419,7 +419,7 @@ def set_image_url(cls, values):
"""Set image_url before rendering the model."""
project_id = values.id
if project_id:
image_dir = f"images/{project_id}/screenshot.png"
image_dir = f"projects/{project_id}/map_screenshot.png"
values.image_url = get_image_dir_url(settings.S3_BUCKET_NAME, image_dir)
return values

Expand Down
5 changes: 2 additions & 3 deletions src/backend/app/tasks/task_logic.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,6 @@ async def get_task_geojson(db: Connection, task_id: uuid.UUID):
status_code=HTTPStatus.NOT_FOUND, detail="Task not found"
)
return data[0]
# return json.loads(data[0]["geom"])


async def update_task_state(
Expand Down Expand Up @@ -123,8 +122,8 @@ async def request_mapping(
"task_id": str(task_id),
"user_id": str(user_id),
"comment": comment,
"unlocked_to_map_state": initial_state.name, # State.UNLOCKED_TO_MAP.name,
"request_for_map_state": final_state.name, # State.REQUEST_FOR_MAPPING.name,
"unlocked_to_map_state": initial_state.name,
"request_for_map_state": final_state.name,
},
)
result = await cur.fetchone()
Expand Down
56 changes: 33 additions & 23 deletions src/backend/app/tasks/task_routes.py
Original file line number Diff line number Diff line change
Expand Up @@ -158,7 +158,7 @@ async def task_states(
db: Annotated[Connection, Depends(database.get_db)], project_id: uuid.UUID
):
"""Get all tasks states for a project."""
return await task_schemas.TaskState.all(db, project_id)
return await task_schemas.Task.all(db, project_id)


@router.post("/event/{project_id}/{task_id}")
Expand All @@ -177,38 +177,44 @@ async def new_event(
project = project.model_dump()
match detail.event:
case EventType.REQUESTS:
if project["requires_approval_from_manager_for_locking"] is False:
data = await task_logic.request_mapping(
db,
project_id,
task_id,
user_id,
"Request accepted automatically",
State.UNLOCKED_TO_MAP,
State.LOCKED_FOR_MAPPING,
# Determine the appropriate state and message
is_author = project["author_id"] == user_id
requires_approval = project["requires_approval_from_manager_for_locking"]

if is_author or not requires_approval:
state_after = State.LOCKED_FOR_MAPPING
message = "Request accepted automatically" + (
" as the author" if is_author else ""
)
else:
data = await task_logic.request_mapping(
db,
project_id,
task_id,
user_id,
"Request for mapping",
State.UNLOCKED_TO_MAP,
State.REQUEST_FOR_MAPPING,
)
# email notification
state_after = State.REQUEST_FOR_MAPPING
message = "Request for mapping"

# Perform the mapping request
data = await task_logic.request_mapping(
db,
project_id,
task_id,
user_id,
message,
State.UNLOCKED_TO_MAP,
state_after,
)
# Send email notification if approval is required
if state_after == State.REQUEST_FOR_MAPPING:
author = await user_schemas.DbUser.get_user_by_id(
db, project["author_id"]
)
html_content = render_email_template(
template_name="mapping_requests.html",
template_name="requests.html",
context={
"name": author["name"],
"drone_operator_name": user_data.name,
"task_id": task_id,
"project_id": project_id,
"project_name": project["name"],
"description": project["description"],
"FRONTEND_URL": settings.FRONTEND_URL,
},
)
background_tasks.add_task(
Expand All @@ -217,6 +223,7 @@ async def new_event(
"Request for mapping",
html_content,
)

return data

case EventType.MAP:
Expand All @@ -233,16 +240,18 @@ async def new_event(
db, requested_user_id
)
html_content = render_email_template(
template_name="mapping_approved_or_rejected.html",
template_name="approved_or_rejected.html",
context={
"email_subject": "Mapping Request Approved",
"email_body": "We are pleased to inform you that your mapping request has been approved. Your contribution is invaluable to our efforts in improving humanitarian responses worldwide.",
"task_status": "approved",
"name": user_data.name,
"drone_operator_name": drone_operator["name"],
"task_id": task_id,
"project_id": project_id,
"project_name": project["name"],
"description": project["description"],
"FRONTEND_URL": settings.FRONTEND_URL,
},
)

Expand Down Expand Up @@ -277,14 +286,15 @@ async def new_event(
db, requested_user_id
)
html_content = render_email_template(
template_name="mapping_approved_or_rejected.html",
template_name="approved_or_rejected.html",
context={
"email_subject": "Mapping Request Rejected",
"email_body": "We are sorry to inform you that your mapping request has been rejected.",
"task_status": "rejected",
"name": user_data.name,
"drone_operator_name": drone_operator["name"],
"task_id": task_id,
"project_id": project_id,
"project_name": project["name"],
"description": project["description"],
},
Expand Down
Loading
Loading