Refactor data fetching and filtering logic in get.ts

[dnafication]

Refactor data fetching and filtering logic in get.ts

This pull request refactors the data/get.ts file to make it more readable and modular. The data fetching logic is now separated from the filtering logic, and reusable utility functions have been extracted into a separate utils.ts file.

The changes include:

• Renamed data/get.ts to data/index.ts and converted it to use async/await syntax
• Extracted data fetching logic for github and gitlab into separate functions into their own files
• Common processing logic lives in the shared.ts file
• Added a new Data interface to define the shape of the data file to be written
• Extracting reusable utility functions into a separate utils.ts file
• Perform write operations for the JSON data file and sitemap data in parallel asynchronously
• Logs all the tags and languages that are being excluded by the filter
• Updating function and variable names to be more descriptive
• Simplifying and optimizing code where possible
• Added descriptive comments to functions and variables

These changes should make the code easier to read, maintain, and extend in the future.

Please review and let me know if there are any issues or concerns with these changes. Happy to discuss and make any necessary changes.

related to #206 and its discussions

[vercel]

Someone is attempting to deploy a commit to a Personal Account owned by @lucavallin on Vercel.

@lucavallin first needs to authorize it.

[lucavallin]

Great work - thanks for the refactoring!

[dnafication]

Thanks for the comments, I'll update the filename and the other comments tomorrow and then merge it.

…

On Mon, Oct 9, 2023, 8:23 PM Luca Cavallin ***@***.***> wrote: ***@***.**** approved this pull request. Great work - thanks for the refactoring! — Reply to this email directly, view it on GitHub <#229 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABQWAE77XOIZNZKFJQNT7LLX6O7CRAVCNFSM6AAAAAA5XMWDDGVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMYTMNRUGA2DCMRTGE> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[dnafication]

@lucavallin Reviews comments have been addressed and the code is ready for merging. Please let me know if there are any additional concerns.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
first-issue	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Oct 11, 2023 8:01pm

[lucavallin]

@dnafication Sorry for the delay! Approved, but there is a conflict.

[dnafication]

Resolved conflicts and merged the latest main. Feel free to merge it @lucavallin

[lucavallin]

🥳

Thanks!