Skip to content

Sprint Reviews

Jump to bottom
thomas loubrieu edited this page Nov 14, 2024 · 107 revisions

Sprint Ending 2024-11-14

Hot map: 20200313_retouch11

Demos

  • Web portal updates (Anil, Eddie, Sean):
    • improvments to MVP
    • drupal
  • peppi (Scott)
  • Rings ontology (Sara, Jordan)
  • Information model upgrades for build 15.0 (Steve, Jordan)
  • New registry tools release and node on-boarding status (Al, Alex, Sagar, Thomas)
  • Harvest in nucleus (Ramesh)
  • SBN Catch foreseen deployments (Vivian)

Sprint Ending 2024-10-24

Demos

  • Localstack for DUM (Scott)
  • Registry, new release after beta test phase (Thomas, Al)
  • TRR (Vivian)
  • Testrail, more automation (Thomas)
  • Data curation, migration for the Web Mod portal (Jordan, Sean, Thomas)
  • Web Mod Portal, MVP +14 days (Anil, Eddie)

Sprint Ending 2024-10-03

Demo

Sprint Ending 2024-09-12

Demo

  • Web modernization:

    • Search (Eddie)
    • Home page (Anil)
    • Deployment on MCP Dev ? (Sean, Eddie)

  • deep-archive bug fixes (Sean)

  • Cognito service for real users (Ramesh)

  • registry-sweeper for multi-tenant registry (Alex, Sagar)

  • API doc, clients updates (Thomas)

  • Seismic use case (Thomas)

  • Discipline node registry beta test (Thomas)

Sprint Ending 2024-08-22

Demo

Sprint Ending 2024-08-01

Demos

  • AWS Cost management per node (Ramesh)
  • Cognito based authentication for Nucleus (Ramesh)
  • --S3 metadata updated by Data Upload Manager (Scott)--
  • cloud front deployment with terraform (Thomas, Vivian, Sagar)
  • local stack, a solution for integration tests ? (Scott)
  • registry manager can set archive status and delete data (Al)
  • registry API, latest development and non-regression automated tests, credentials renewals (Alex, Thomas)
  • New continuous monitoring (Sean)
  • GIS web demo with new layers, QGIS export (Matt)
  • automated tests for PDS4 Information Model and LDDTool (Miguel, Jordan)
  • Updated context product for web modernization (Steve, Sophia)
  • web modernization updates (Anil, Eddie)

Sprint Ending 2024-07-11

Demos

  • Status on registry migration, from registry user workshop (Thomas)
  • Issue with OpenSearch serverless (Al, Thomas)
  • s3-browser on non-root URLs (Eddie)
  • Editorial working group, what is it ? (Jordan, Mike, Ed - Jordan will do this presentation if the MC has ended in time)
  • Web modernization updates (Anil, Eddie)
  • GIS Treks Titan (Matt)

Sprint Ending 2024-06-20

Demos

  • new test data for deep archive (Sean)
  • DUM updates (Scott)
  • Nucleus updates (Ramesh)
  • Registry loader for multi-tenant registry, docker image (Thomas)
  • GIS internship introduction (Matt)
  • Authentication for Opensearch serverless and Java clients (Sagar)
  • Command line tool to query the multitenant registry (Alex)
  • terraform script for registry-api on MCP (Thomas)
  • web modernization updates (Anil, Eddie)

Sprint Ending 2024-05-30

Demos

  • DDR (Jordan)
  • Web modernization (Eddie)
  • Continuous Deployment in Dev venue (Sean)
  • Continuous Integration for web development (Eddie based on Sean development)
  • Validate-ref and other validate updates (Al)
  • Nucleus multitenant (Ramesh)
  • Multitenant registry migration (Thomas, Sagar, Al, Alex)
    • infrastructure (Sagar)
    • opensearch/data (Sagar)
    • harvest/registry-mgr (Al, Thomas)
    • registry-sweeper (Alex)
    • command line tool (Alex)
    • Search API (Thomas)

Sprint Ending 2024-05-09

Demos

  • DUM upgrades (Scott)
  • Nucleus automated deployment (Ramesh)
  • Multitenant registry
    • Plan/schedule/communication with the nodes (Jordan, Thomas)
    • Data migration (Sagar)
    • harvest/registry-mgr (Al)
    • sweeper (Alex)
    • api (Thomas)
  • NASA SMD Software workshop, API lightning talk and overall report (Thomas)
  • PDS4 Information Model upgrades (Steve)
  • Web analytics pipelines improvments (Kai)
  • Web modernization implementation (Anil, Eddie)

Sprint Ending 2024-04-18

Demos

  • Baseline IAM policies (Sagar)
  • Data Upload Manager upgrades (Scott)
  • Loader tools (harvest/registry-mgr) for multi-tenant registry (Al)
  • Nucleus updates (Ramesh)
  • S3 browser (Eddie)
  • Web modernization implementation updates (Anil, Eddie)
  • NASA A&A (Jordan)
  • Keyword search on legacy registry (Jordan)

Sprint Ending 2024-03-28

Demos

  • Continuous Integration for nodejs, on dummy repository (Sean)
  • LDD bug fixes (Steve, Jordan)
  • Nucleus updates (Ramesh)
  • Multitenant updates (Sagar, Al)
  • Registry-api, integration of new OpenSearchClient, first step (Thomas)
  • Web modernization implementation updates (Anil, Eddie)

Sprint Ending 2024-03-07

Demos (5–10 minutes each)

  • Upload CSS data on the cloud with DUM (Vivian, Ramesh)
  • add deep-archive client to registry-api branch CICD test (Jordan)
  • CodeQL static analysis (Jordan)
  • Summer Internship on GIS (Thomas)
  • Web modernization implementation status (Anil, Eddie)

Bug fixes (1 minute each)

  • Registry-Sweepers improvements, memory, ... (Alex)
  • Deep-archive bug fix and api dependency upgrade (Sean)
  • RDD automated generation fix (Thomas, Jordan)
  • 500 error on registry-api fix (Alex)
  • MI-label bug fix (Jordan)

Status/Planning, mostly planning (20–30 minutes each, as needed)

  • Nucleus (Ramesh, all) 20'
    • sprint objectives:
      • DUM load to S3 staging bucket and synchronize to ODR s3 bucket
      • (out of scope for now) at bundle level: pds-deep-archive, registry-mgr release.
      • DAG at product level:

For CSS:

  1. validate
  2. validate-ref (on-hold)
  3. harvest
  4. glacier
  5. copy data to ODR
  6. remove data from EFS
  7. remove staging s3 bucket
  • Multitenant Registry (Sagar, Al, all) 20'

Sprint Ending 2024-02-15

Demos

  • Open Data Registry, what is it ? How is it useful for PDS/CSS (Vivian)

  • Incident Response Plan (Jordan)

  • Build 14.1 plan update submitted to Software Working Group (Jordan)

  • Validate on Windows (Jordan)

  • Multitenants Registry updates (Sagar)

  • Documentation of archive status for registry (Thomas)

  • Nucleus udpdates (Ramesh)

  • property list in pds-api-client (Thomas)

  • Web modernization updates (Anil, Eddy)

  • What are your takes on the Software Symposium (all)

  • Other

Sprint Ending 2024-01-25

Recording: TBD

Demos

  • PDS Engineering Node web site migrated and 508 compliance (Catherine)

  • Archive Browser with AWS SDK, progress so far (Eddie)

  • New web design implementation, progress so far (Anil)

  • Nodejs repo template and roundup integration (Sean)

  • CORS issues on feedback widget (Thomas)

  • Harvest updated job configuration for multi-tenant opensearch (Al, Jordan, Thomas)

  • Nucleus documentation on gh-pages (Ramesh):

Planning

Web implementation sprint to be detailed

Sprint Ending 2023-12-14

Recording: https://jpl365prod-my.sharepoint.com/:v:/g/personal/jordan_h_padams_jpl_nasa_gov/EXzQ04eWiWRDm3BFugAaapUBYusRLpK8prID-tTgcr0OKQ?referrer=Teams.TEAMS-ELECTRON&referrerScenario=MeetingChicletGetLink.view.view

Demos

  • Press It Up: Web modernization development initial setup with Wordpress (Anil)
  • Buckets for Days: Latest greatest web directory page for S3 buckets (Eddie)
  • Paging 1995: New pagination interface and implementation for the registry-api (Alex)
  • Sweepers Creepers: Consistent properties as list vs string in registry with sweepers (Alex)
  • Checking In: Review the latest updates to Validate (Jordan)
  • State of Nucleus Address: Latest Nucleus Updates and "end-to-end" execution (Ramesh)
  • Fresh Rolls: Best Practices in creating IAM Roles and Security Groups (Sagar)
  • Registry Design-a-thon: Multi-tenant Registry Architecture Updates for supporting signed URLs (Sagar)

Sprint Ending 2023-11-23

Meeting held on 2023-11-30 due to holiday Recording: https://jpl365prod-my.sharepoint.com/:v:/g/personal/jordan_h_padams_jpl_nasa_gov/EYw3imfWNp9IlGgVIQqJk4MBv79cyNb5NvQJp39JQDi7Xg?referrer=Teams.TEAMS-ELECTRON&referrerScenario=MeetingChicletGetLink.view.view

Demos

  • Web modernization design update (Mike/Edward)
  • API wrapper PoC (Thomas)
  • NAIF node missing in legacy dashboards (Thomas)
  • Nucleus with fitz files (compressed), Nucleus cost issue and alternate design proposal (Ramesh)
  • Re-try behavior on harvest (Alex)
  • Build 14.0 I&T, TRR, DDR (Vivian, Miguel, Gary, Thomas)
  • Don't leak your secrets (Sean)

Sprint Ending 2023-11-02

Recording: https://jpl365prod-my.sharepoint.com/:v:/g/personal/jordan_h_padams_jpl_nasa_gov/EYrNv4VEvKJHnIuSHzr6ulwBpFny9mCsMCKcHyJU49qBjw?referrer=Teams.TEAMS-ELECTRON&referrerScenario=MeetingChicletGetLink.view.view

Demos:

Legacy

  • Updates on MCP (Sagar)
    • Encryption of EC2 instances
    • OIDC service
  • Data Upload Manager status and documentation (Scott)
  • Nucleus integration with Data Upload Manager (Ramesh), successful workflow with Messenger as planned for the CCS meeting
  • First iteration of the provenance model in PDS4 (Steve)
  • Release note with known bugs, for example https://github.com/NASA-PDS/registry-api/releases/tag/v1.3.1 (Thomas)
  • Deep archive, installation instructions for Windows (Sean)

Sprint Ending 2023-10-12

Recording: TBD

Demos:

Data curation

  • Cutover to using new Registry API for generating context products json (Al, Jordan): cancelled, not working yet because API is slow over a few pages of results.
  • Avoid race condition in the product checker (Ramesh)

Search and web

  • DPS presentations (Jordan: slides, Thomas: poster)
  • Sinequa useability analysis (Joe)
  • Navigation app bar in new web (Anil)

Other

  • Create a stable release for the registry, what does that mean ? (Thomas)
  • ISA status (Jordan)

Crowning piece

  • Web Prototype (Mike)

Sprint Ending 2023-09-21

Demos:

  • Web analytics (Kai)
  • Web portal architecture with WordPress (Anil)
  • Vulnerability fix from dependabot alerts on web development (Anil)
  • Request the Sinequa API for search (Joe)
  • Registry Automated Tests maintenance (Thomas)
  • Registry-sweepers/RepairKit tuning for production (Alex, Sagar)
  • DUM: Infrastructure as Code (Scott)

Sprint Ending 2023-08-31

Recording: TBD

Demos:

  • Product Completion Checker lambda for Nucleus (Ramesh)
  • New LDD (LRO, mgn) (Catherine, Steve)
  • RepairKit optimization for production (Alex)
  • Registry loading status dasboard in Kibana (Thomas)
  • CloudWatch log monitoring for Registry-Sweepers & Registry-API using AWS Lambda / SNS (Sagar)
  • Messenger data visualization with the PDS Web API and LeafMap (Joaquin)
  • IPDA new web site published, the steps between after development to publication (Sean, video link (JPL only), PDF of slides)
  • DUM: Parallelization of upload (Scott)

Sprint Ending 2023-08-10

Recording: TBD

Demos:

  • The balkanization of pds-github-utils (Sean)
  • Add TermMaps to default output JSON (Steve)
  • Harvest tags the loaded products with its version number (Al, Thomas)
  • Performance improvement on registry-sweepers (Alex)
  • New alerts on registry-api and sweepers (Sagar)
  • Nucleus, workflow triggers (Ramesh)
  • Registry-api, JSON for everyone (Al, Thomas)
  • Read images with the registry-api (Joaquin)
  • Published results on the LLM tests for PDS Search (Amanda)

Sprint Ending 2023-07-20

Recording: https://jpl365prod-my.sharepoint.com/:v:/r/personal/thomas_g_loubrieu_jpl_nasa_gov/Documents/Recordings/PDS%20Sprint%20Review-20230720_141111-Meeting%20Recording.mp4?csf=1&web=1&e=KTgX2m&nav=eyJyZWZlcnJhbEluZm8iOnsicmVmZXJyYWxBcHAiOiJTdHJlYW1XZWJBcHAiLCJyZWZlcnJhbFZpZXciOiJTaGFyZURpYWxvZyIsInJlZmVycmFsQXBwUGxhdGZvcm0iOiJXZWIiLCJyZWZlcnJhbE1vZGUiOiJ2aWV3In19

Zenhub says: image

Demos:

  • Updated npm package architecture for web UIs (Eddie, Anil)
  • Updates and plans for Magnetometers analysis from the registry-api (Joaquin)
  • Semantic Search/LLM, similarities results (Amanda)
  • Analysis for product-triggered workflow for nucleus (Ramesh)
  • Standards for describing AirFlow DAGs, options (Ramesh)
  • Alerts on almost full storage on AWS (Sagar)
  • Registry Docker compose for multiple architectures (Sean); presentation available as video (JPL-only), or as PDF

Sprint Ending 2023-06-29

Recording: https://jpl365prod-my.sharepoint.com/:v:/r/personal/thomas_g_loubrieu_jpl_nasa_gov/Documents/Recordings/PDS%20Sprint%20Review-20230629_143149-Meeting%20Recording.mp4?csf=1&web=1&e=NkeQbS

  • Continuous Deployment with terraform on NGAP (Sagar) 5′
  • Registry-api postman tests reports in TestRail (Sean) 5′
  • Updated Sitemap for PDS web site (Catherine, Sean) 5′
  • Use signed-url/cognito to connect to opensearch (Alex) 5′
  • Registry-sweeper, what it does, how it is deployed (Alex) 5′
  • Registry-api OVIRS notebook, OVIRS data loaded and new pds.api-client (Thomas) 5′
  • Registry-api notebook: plot Galileo magnetometer observations extrcted from the registry-api with pyWWT (Joaquin) 5′
  • Data Upload Manager: monitor the client commands launched by the Discipline Nodes (Scott) 5′
  • Validate validates video labels and files (Al, Jordan) 5′
  • Registry loader procedures documented , see https://github.com/NASA-PDS/operations/issues/359 (Vivian) 5′
  • Semantic Search: Compute embedding vectors on PDS4 labels and user's search term, status and plan for next sprint (Amanda) 5′
  • Sinequa Search (Joe) 5′
  • Other:

Open ticket/planning

  • jenkins vs harness (vs github actions) ?
  • Messenger data loaded by Nucleus/Ramesh (to be used by Joaquin)
  • DOI-Editor, soon available in production, what are the next steps ?
  • Schedule a point build and deployment for registry-api (after bug fixes)
  • Search architecture, how to move forward, sinequa/opensearch/semantic search/graph database (https://github.com/NASA-PDS/portal-tasks/issues/72) ...
  • Start archive viewer development

Sprint Ending 2023-06-08

Recording: https://jpl365prod-my.sharepoint.com/:v:/r/personal/jordan_h_padams_jpl_nasa_gov/Documents/Recordings/PDS%20Sprint%20Review-20230608_142204-Meeting%20Recording.mp4?csf=1&web=1&e=sbjoHr

Demos

List to be completed the morning of the Sprint review.

The demos are 5 minutes max but less is also ok, with short word on something which was completed, a screenshot, an explanation of what it is useful for:

  • Latest development of the Search Prototype (Eddie, presented by Thomas) 5'

  • LLMs for PDS, internship plan (Amanda) 5'

  • Nucleus (Ramesh): 5' [] share data between steps [] logging organization and presentation to users/operators

  • New test dataset for Registry integration tests (Thomas) 5'

  • Registry-api notebooks reaching our user's community (Joaquin) 5'

  • New health check end-point (Jimmie) 5'

  • Miscellaneous, 1 screenshot each: 5'

    • B13.1 software release (Thomas)
    • JPL logo on PDS ENG wordpress (Catherine)
    • Organize a happy hour and hurt oneself so not to be sure to attend (Jordan)

Pushed to next sprint review because of training or sickness:

  • Registry-sweeper, what it does, how it is deployed (Alex) 5'
  • Continuous Deployment with terraform on NGAP (Sagar) 5'
  • Registry-api postman tests reports in TestRail (Sean) 5'

Other significant blocked tasks

Detailed review of the remaining open ticket by Thomas and Jordan. Next sprint only blockers should be raised by the team. Thomas and Jordan will review the remaining tickets offline or in a different meeting.

Copyright © 2021-2024 California Institute of Technology.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Clone this wiki locally