Log repositories not matching requirements

[zillemarco]

Since we have multiple reasons why a project is not added (namely stars count and issues count) and we have different places where we do filter those out (for GitHub is a mix of GraphQL query and manual filter, for GitLab is only manual filter).

Probably, a good way to handle this would do all the queries and then iterate over the repos coming from a source and see if it's there or not, and log those results.

Originally posted by @zillemarco in #171 (comment)

[dnafication]

Hey @zillemarco, sounds interesting. It will definitely simplify the filtering process. Where would you like that to be logged?

[zillemarco]

Do you mean where to create the log file with the actual logs, or where to log the data inside the code? 🤔

For the file, I thing we can stick with console.log as it's already used. For where to do the logging I think the best place would be on the last execution block inside data/get.ts, right before the sitemap build 🤔

But I'm sure @lucavallin would be able to tell you if that's ok or if there are better ways/places 🙂

[lucavallin]

@dnafication agree with @zillemarco. We just need a way to see repos that don't meet criteria (anymore). Maybe in the Actions workflow summary would be a reasonable place for that.

[dnafication]

Great! I have some ideas around refactoring the get.ts file and making it modular and more readable. I'll include the logging changes there. I'll send a PR soon.

[dnafication]

Hey @lucavallin @zillemarco, I was thinking of introducing .env file to the project and add all these vars below and more to the file.

REPOS_PER_REQUEST=25
MAX_ISSUES=10
GH_PERSONAL_ACCESS_TOKEN=
... any others in the future

@lucavallin you might need to update Vercel's deployment config to support these changes so let me know if its worth a change.

[dnafication]

At first, my idea was to record comprehensive information regarding why a repository didn't meet the criteria. However, it appears that filtering occurs at various stages, such as during the GraphQL query and subsequent processing of its results.

Now, I'm planning to generate a comparison between the original list and the filtered list and logging only that information. Do you believe this would suffice, or should we still include specifics about why a particular repository failed to meet the criteria?

Perhaps, this can be the initial iteration and we can add more logic to it. I have made the code modular and refactored the main script to make it readable. I'll send the PR soon.

[lucavallin]

@dnafication I am happy to add the .env file, but first, what problem does it solve?
For the rest, all good. I think a list of repositories filtered out without details is fine for now.