Feature request: changelog metric

[eccegordo]

One of the technical hurdles that plagues a lot of novices is the dependency management hurdles when upgrading.

To that end it would be great if Ember Observer could help incentivize addon community to put more effort into coherent changelogs.

Ember CLI is the gold standard for this with their releases page.

https://github.com/ember-cli/ember-cli/releases

Other ideas:

• compatibility matrix in the readme.
• known deprecations (like on the ember website)
• errata/known issues (consolidated issues sign posts to people upgrading)

Not sure what the metric would look like exactly but perhaps there is a common changelog format that could be used that would give the addon project more weight if they adopted it.

Also, curious if some of the ember try functionality could be leveraged to enforce consistency checks between addons and automate some of this.

Thanks!

[kategengler]

We've considered using the presence of a Changelog as point. Format and content would be a bit hard to manage -- we try to keep everything automated, and while format might be possible to automate, it would probably be resource-intensive.

Ember Observer already does run ember-try scenarios for the top 100 addons, in an attempt to generate compatibility matrices for them. We have plans to extend that to all addons.

rwjblue turned me on to https://github.com/skywinder/github-changelog-generator to generate Changelogs. It's pretty great but is a Ruby gem so might not be the most convenient thing for maintainers that don't already have Ruby installed.

[eccegordo]

Yeah good point about the automation. Maybe the answer is some clearly identified hook or well defined pattern of content/format that is easy for ember observer to detect and parse.

Maybe even publish the changelog on the ember observer website, like you do the readme.

I am not too concerned about the exact content, or "gaming the ratings". Have to trust people will do the right thing. Giving it visibility seems sufficient to encourage good behavior. Main point being asking people to put at least some thought into releasing a changelog feels like a win. What is really discouraging is seeing projects with only release versions and nothing else. (liquid-fire I am looking at you)
edit: I see now liquid-fire has a CHANGELOG.md file but nothing on GitHub releases page.

Have you given thought to anything that can subtract from the score? Meaning I do all these things correctly but do this one thing bad and there are negative consequences. Carrot and stick.

I hate to pick on liquid-fire because I like the library and people behind it. But I am not sure it deserves a perfect score score because in my experience it is one the first libraries to have issues whenever I try to upgrade CLI. The lockstep concept isn't quite there yet.

Anyway, thanks for the changelog-generator protip, and for your consideration and time.

Feel free to close this issue if there is nothing immediately actionable at the present time.

[kategengler]

Parsing the changelog file for each addon for a particular format would be a bit much. Checking for the presence of a changelog is definitely reasonable though.

We considered subtracting from the score but disregarded the idea. If we add more criteria, there will be points for doing them correctly but not removed for doing things the "wrong" way. Those addons will still have a lower score than others.

So, liquid-fire would get our point for having a changelog, because here it is https://github.com/ember-animation/liquid-fire/blob/master/CHANGELOG.md If you're having problems when upgrading with liquid-fire, I suggest you file an issue.