Replace github-linguist gem with Node port Nixinova/Linguist

[Nixinova]

Description

Using the github-linguist gem directly is a bit of a hack, and any changes to console.log output breaks Metrics, like what happened with the release of v7.16.

Nixinova/Linguist is a Node port of github-linguist which allows for proper JavaScript language data generation.

This change also allows for prose and data languages to be included in any generated metrics as a new option if so desired.

Tests

• Generated with metrics@master
• Generated with this change

Refs

• Fully resolve Languages plugin no longer works #424
• See also: Add option to configure which Linguist categories to display #341

[lowlighter]

Thanks for your contribution!
It'll definitely improve stability of languages plugins 👍

[Nixinova]

Since the output of Linguist is this ↓, is there anywhere in the languages plugin where language processing can now be simplified?

Linguist output schema

{
	"count": 3,
	"results": {
		"src/index.ts": "TypeScript",
		"src/cli.js": "JavaScript",
		"readme.md": "Markdown"
	},
	"languages": {
		"all": {
			"JavaScript": { "type": "programming", "bytes": 1000 },
			"TypeScript": { "type": "programming", "bytes": 2000 },
			"Markdown": { "type": "prose", "bytes": 3000 },
		},
		"programming": { "JavaScript": 1000, "TypeScript": 2000 },
		"markup": {},
		"data": {},
		"prose": { "Markdown": 3000 },
		"unknown": {},
		"total": { "unique": 3, "bytes": 6000, "unknownBytes": 0 }
	}
}

For example I see a couple instances of looped 'results[lang].bytes+=size':

metrics/source/plugins/languages/index.mjs

Lines 47 to 52 in 3e26d37

	//Process repository languages
	for (const {size, node:{color, name}} of Object.values(repository.languages.edges)) {
	languages.stats[name] = (languages.stats[name] ?? 0) + size
	languages.colors[name] = colors[name.toLocaleLowerCase()] ?? color ?? "#ededed"
	languages.total += size
	}

(I could also add a color key to languages.all[lang] if it would be useful.)

[lowlighter]

This loop is for GitHub api data, not for linguist data. It's the non-indepth mode so it cannot be removed.

Linguist is only called in async function analyze() from analyzers.mjs, and I don't think we can simplify it further since once linguist data is gathered, that's pretty much it:

metrics/source/plugins/languages/analyzers.mjs

Lines 140 to 142 in 63675f1

	//Gather language data
	console.debug(`metrics/compute/${login}/plugins > languages > indepth > running linguist`)
	const {results:files, languages:languageResults} = await linguist(path)

Adding the color key would help to close #339 though, since it'll help covering languages that are not loaded from GitHub api and not user-defined

[lowlighter]

I'll make a release soon even if there are still improvements that can be made, since #424 still affects @latest release

[Nixinova]

I also think the problem in #356 may be covered by my linguist.

[Nixinova]

Adding the color key would help to close #339 though, since it'll help covering languages that are not loaded from GitHub api and not user-defined

Added in 1.4.4: get from .languages.all[lang].color

[lowlighter]

I also think the problem in #356 may be covered by my linguist.

This PR was for gathering patches from GitHub events along with their associated repository .gitattributes. Since it's mostly for code collection I think it's unrelated though I may be wrong.

Added in 1.4.4: get from .languages.all[lang].color

Nice 👍