[C4GT Community]: Implement ELK Stack for Comprehensive Observability in AMRIT Platform

[drtechie]

Ticket Contents

Description

AMRIT (Accessible Medical Records via Integrated Technologies) is a digital health initiative by Piramal Swasthya, aimed at integrating healthcare services and providing accessible medical records. To ensure the platform's reliability and efficiency, we need to implement a robust observability solution using the ELK stack (Elasticsearch, Logstash, Kibana). This solution will capture logs and traces from our Spring Boot services, enabling real-time monitoring, alerting, and troubleshooting.

What You Will Learn:

As an intern, you will:

• Gain hands-on experience with the ELK stack and observability practices.
• Learn how to integrate logging and tracing in Spring Boot applications.
• Develop skills in configuring and optimizing Elasticsearch for log storage and analysis.
• Work with Kibana to create dashboards and visualizations for monitoring application health.
• Collaborate with a team of experienced engineers and gain insights into the healthcare technology sector.

Desired Skills:

• Basic knowledge of Java and Spring Boot.
• Familiarity with logging frameworks.
• Understanding of RESTful APIs and microservices architecture.
• Enthusiasm for learning new technologies and frameworks.
• Comfortable using Git/GitHub for version control.

Goals

• Set up ELK Stack - configure Elasticsearch, Logstash, and Kibana on the development environment; integrate the ELK stack with existing Spring Boot services.
• Logging Integration - implement logging in Spring Boot services; parse and ship logs to Elasticsearch.
• Tracing Implementation - set up Elastic APM and enable tracing in Spring Boot services; visualize traces in Elastic Stack for end-to-end request tracking.
• Midpoint Goals - create basic dashboards to monitor service logs and performance metrics; ensure logs are searchable and traces are correctly linked per project/environment.
• Advanced Monitoring - enhance dashboards with custom visualizations for error rates, latency, and traffic; set up alerts for critical issues.

Expected Outcome

A fully functional observability solution for the AMRIT platform, allowing the operations team to monitor logs and traces in real time. The ELK stack should be well-integrated with AMRIT's API services, providing clear insights into the platform's performance, with a focus on ensuring system reliability and quick issue resolution.

Acceptance Criteria

• Logs from all Spring Boot services are indexed in Elasticsearch and accessible via Kibana.
• Traces are correctly captured and visualized in Elastic Stack, showing the flow of requests across services.
• Logs and traces are segregated based on deployments.
• Role-based access to ELK stack.
• Good to have: Alerts are configured for critical log patterns and performance anomalies.

Implementation Details

The observability solution will be built using the ELK stack, with logging and tracing integrated into existing Spring Boot services. You will get access to a server with ELK services set up by our IT team.

Mockups/Wireframes

NA

Product Name

AMRIT

Organisation Name

Piramal Swasthya Management Research Institute

Domain

Healthcare

Tech Skills Needed

Debugging, DevOps, Java, RESTful APIs, Spring Boot, SQL

Mentor(s)

@drtechie

Complexity

High

Category

Backend, Deployment, Performance Improvement

How to apply

Please fill the form.

Bounty

This is a bounty ticket with a remuneration of INR 10,000.

[ivor11]

Please assign this issue to me