SPIKE: Investigate fixing comparison tool

[dustinsgoodman]

Background

Deno and other language libraries aren't on the NPM registry. We want to still be able to compare these tools. This ticket is a SPIKE to explore options to solve the library compare feature for other tools.

Questions to Answer

• Can we support Deno like we support NPM?
• On the GraphQL site, we added other non-JS tools. Can we support some of these other languages? Python has PIP, Ruby has gems, etc. Can we integrate with these to pull the same stats?

[dariodjuric]

Current state

Framework.dev currently shows the following categories in the comparison tool and works only for npm-based libraries:

• Testing Coverage
• Weekly Downloads
• Overall Health => calculated by npms.io by various parameters such as existence of a readme, existence of tests, version stability...
• Stars

The above information for each library is retrieved from npms.io.

Adding support for other types of libraries

If we were to support other types of libraries (non-npm), then we would need to consider our options per language, since there is no single site that offers stats for different languages. Stats too vary, depending on what we can get via APIs. Testing coverage, for example, is not something that is easily retrievable. Below are options that we can consider for PHP, Ruby, Python, and Deno.

PHP

In PHP, Composer is most often used for managing dependencies. packagist.org is a repository of PHP packages, and they offer an API to get package stats. For example:

// GET https://packagist.org/packages/[vendor]/[package].json

{
  "package": {
    "name": "[vendor]/[package],
    "description": [description],
    "time": [packagist package creation datetime],
    "maintainers": [list of maintainers],
    "versions": [list of versions and their dependencies, the same data of composer.json]
    "type": [package type],
    "repository": [repository url],
    "downloads": {
      "total": [numbers of download],
      "monthly": [numbers of download per month],
      "daily": [numbers of download per day]
    },
    "favers": [number of favers]
    ...
  }
}

If the package is hosted on GitHub, additional properties are returned:

    "github_stars": X,
    "github_watchers": Y,
    "github_forks": Z,
    "github_open_issues": M,

Working example: https://packagist.org/packages/monolog/monolog.json

Some stats we could consider for framework.dev in the context of Packagist:

• package.downloads.monthly
• package.github_stars
• package.favers (see below)
• package.dependents (see below)
• package.suggesters (see below)

Favers refers to the number of users who have marked a package as a favorite on the Packagist website. This feature allows users to keep track of packages they find useful or interesting, and can also be used as a way to discover new packages that have been well-received by the community. The number of "favers" for a package can be an indication of its popularity and usefulness.

Dependents in Packagist refers to the number of other packages that depend on a given package. This means that the dependent packages have specified the package as a requirement in their own composer.json file and thus have a dependency on it.

Suggests is a way for package maintainers to suggest other packages that may be used with the package they are publishing, but are not required for it to function. It's a way to give users a heads up of what else they might find useful. Suggesters is a number of package package maintainers that have suggested this package.

Ruby

In Ruby, RubyGems.org is the index of Ruby gems (packages) and there is also BestGems.org which offers download statistics for Ruby gems.

For example, BestGems.org has the following endpoints for getting downloads and ranking:

• https://bestgems.org/api/v1/gems/[package-name]/total_ranking.json
• https://bestgems.org/api/v1/gems/[package-name]/daily_ranking.json
• https://bestgems.org/api/v1/gems/[package-name]/total_downloads.json
• https://bestgems.org/api/v1/gems/[package-name]/daily_downloads.json

RubyGems.org can get you other information about a package:

// GET https://rubygems.org/api/v1/gems/aws-sdk-core.json

{
  "name": "aws-sdk-core",
  "downloads": 873693539,
  "version": "3.169.0",
  "version_created_at": "2023-01-18T20:31:20.443Z",
  "version_downloads": 691524,
  "platform": "ruby",
  "authors": "Amazon Web Services",
  "info": "Provides API clients for AWS. This gem is part of the official AWS SDK for Ruby.",
  "licenses": [
    "Apache-2.0"
  ],
  "metadata": {
    "changelog_uri": "https://github.com/aws/aws-sdk-ruby/tree/version-3/gems/aws-sdk-core/CHANGELOG.md",
    "source_code_uri": "https://github.com/aws/aws-sdk-ruby/tree/version-3/gems/aws-sdk-core"
  },
  "yanked": false,
  "sha": "81e75e70dfd4a17c55554b593ad2de5534b6cab9197cd8e278eb8b0a9ad4051b",
  "project_uri": "https://rubygems.org/gems/aws-sdk-core",
  "gem_uri": "https://rubygems.org/gems/aws-sdk-core-3.169.0.gem",
  "homepage_uri": "https://github.com/aws/aws-sdk-ruby",
  "wiki_uri": null,
  "documentation_uri": null,
  "mailing_list_uri": null,
  "source_code_uri": "https://github.com/aws/aws-sdk-ruby/tree/version-3/gems/aws-sdk-core",
  "bug_tracker_uri": null,
  "changelog_uri": "https://github.com/aws/aws-sdk-ruby/tree/version-3/gems/aws-sdk-core/CHANGELOG.md",
  "funding_uri": null,
  "dependencies": {}
}

GitHub stars can be retrieved directly from GitHub via the https://api.github.com/repos/[owner]/[project] API endpoint. For example:

// GET https://api.github.com/repos/Seldaek/monolog

{
  "id": 1376664,
  "node_id": "MDEwOlJlcG9zaXRvcnkxMzc2NjY0",
  "name": "monolog",
  "full_name": "Seldaek/monolog",
  "private": false,
  // ...
  "size": 3709,
  "stargazers_count": 20133,
  "watchers_count": 20133,
}

For framework.dev, we might consider:

• Daily ranking
• Daily downloads
• GitHub stars

Python

PyPI is the official package index for Python, and there is also pypistats.org for download statistics.

Downloads can be retrieved from the following endpoint: https://pypistats.org/api/packages/[package-name]/recent

For Django (https://pypistats.org/api/packages/django/recent):

// GET https://pypistats.org/api/packages/django/recent

{
  "data": {
    "last_day": 394453,
    "last_month": 8557361,
    "last_week": 1963861
  },
  "package": "django",
  "type": "recent_downloads"
}

Other package information can be retrieved directly from PyPI via the https://pypi.org/pypi/[package-name]/json endpoint. For example:

// GET https://pypi.org/pypi/django/json

{
  "info": {
    "author": "Django Software Foundation",
    "author_email": "[email protected]",
    "bugtrack_url": null,
    "classifiers": [
      "Development Status :: 5 - Production/Stable",
      "Environment :: Web Environment",
      "Framework :: Django",
      "Intended Audience :: Developers",
      "License :: OSI Approved :: BSD License",
      "Operating System :: OS Independent",
      "Programming Language :: Python",
      "Programming Language :: Python :: 3",
      "Programming Language :: Python :: 3 :: Only",
      "Programming Language :: Python :: 3.10",
      "Programming Language :: Python :: 3.11",
      "Programming Language :: Python :: 3.8",
      "Programming Language :: Python :: 3.9",
      "Topic :: Internet :: WWW/HTTP",
      "Topic :: Internet :: WWW/HTTP :: Dynamic Content",
      "Topic :: Internet :: WWW/HTTP :: WSGI",
      "Topic :: Software Development :: Libraries :: Application Frameworks",
      "Topic :: Software Development :: Libraries :: Python Modules"
    ],
    "description": "======\nDjango\n======\n\nDjango is a high-level Python web framework that encourages rapid development\nand clean, pragmatic design. Thanks for checking it out.\n\nAll documentation is in the \"``docs``\" directory and online at\nhttps://docs.djangoproject.com/en/stable/. If you're just getting started,\nhere's how we recommend you read the docs:\n\n* First, read ``docs/intro/install.txt`` for instructions on installing Django.\n\n* Next, work through the tutorials in order (``docs/intro/tutorial01.txt``,\n  ``docs/intro/tutorial02.txt``, etc.).\n\n* If you want to set up an actual deployment server, read\n  ``docs/howto/deployment/index.txt`` for instructions.\n\n* You'll probably want to read through the topical guides (in ``docs/topics``)\n  next; from there you can jump to the HOWTOs (in ``docs/howto``) for specific\n  problems, and check out the reference (``docs/ref``) for gory details.\n\n* See ``docs/README`` for instructions on building an HTML version of the docs.\n\nDocs are updated rigorously. If you find any problems in the docs, or think\nthey should be clarified in any way, please take 30 seconds to fill out a\nticket here: https://code.djangoproject.com/newticket\n\nTo get more help:\n\n* Join the ``#django`` channel on ``irc.libera.chat``. Lots of helpful people\n  hang out there. See https://web.libera.chat if you're new to IRC.\n\n* Join the django-users mailing list, or read the archives, at\n  https://groups.google.com/group/django-users.\n\nTo contribute to Django:\n\n* Check out https://docs.djangoproject.com/en/dev/internals/contributing/ for\n  information about getting involved.\n\nTo run Django's test suite:\n\n* Follow the instructions in the \"Unit tests\" section of\n  ``docs/internals/contributing/writing-code/unit-tests.txt``, published online at\n  https://docs.djangoproject.com/en/dev/internals/contributing/writing-code/unit-tests/#running-the-unit-tests\n\nSupporting the Development of Django\n====================================\n\nDjango's development depends on your contributions. \n\nIf you depend on Django, remember to support the Django Software Foundation: https://www.djangoproject.com/fundraising/\n\n\n",
    "description_content_type": "",
    "docs_url": null,
    "download_url": "",
    "downloads": {
      "last_day": -1,
      "last_month": -1,
      "last_week": -1
    },
    "home_page": "https://www.djangoproject.com/",
    "keywords": "",
    "license": "BSD-3-Clause",
    "maintainer": "",
    "maintainer_email": "",
    "name": "Django",
    "package_url": "https://pypi.org/project/Django/",
    "platform": null,
    "project_url": "https://pypi.org/project/Django/",
    "project_urls": {
      "Documentation": "https://docs.djangoproject.com/",
      "Funding": "https://www.djangoproject.com/fundraising/",
      "Homepage": "https://www.djangoproject.com/",
      "Release notes": "https://docs.djangoproject.com/en/stable/releases/",
      "Source": "https://github.com/django/django",
      "Tracker": "https://code.djangoproject.com/"
    },
    "release_url": "https://pypi.org/project/Django/4.1.5/",
    "requires_dist": [
      "asgiref (<4,>=3.5.2)",
      "sqlparse (>=0.2.2)",
      "backports.zoneinfo ; python_version < \"3.9\"",
      "tzdata ; sys_platform == \"win32\"",
      "argon2-cffi (>=19.1.0) ; extra == 'argon2'",
      "bcrypt ; extra == 'bcrypt'"
    ],
    "requires_python": ">=3.8",
    "summary": "A high-level Python web framework that encourages rapid development and clean, pragmatic design.",
    "version": "4.1.5",
    "yanked": false,
    "yanked_reason": null
  },

The above response retrieves GitHub repository URL, so this could be used to retrieve GitHub stars.

For framework.dev, we might consider:

• Last month downloads
• Classifiers / Development Status (all possible values are available here here)
• Classifiers / Environment (all possible values are available here here)
• GitHub stars

Deno

Deno distinguishes itself from other languages/frameworks because it doesn't have a centralized package repository. In a Deno app, you can import packages using an URL from anywhere.

Deno native packages are usually listed on deno.land/x. This service provides some basic information about packages, such as repository URL and popularity, but that is about it. It also doesn't provide any API. Probably, all of the information is extracted from the respective GitHub repositories. Popularity, for example, could be calculated by the number of watchers, forks, and stars. We could do something similar for framework.dev.

[dustinsgoodman]

Thank you @dariodjuric! This is a comprehensive list. Some questions I have:

• Do any of these other package managers provide a badging system like NPM? That would be ideal for display purposes otherwise we need to make generic badges for these stats.
• Should we worry about alternative package managers? While PHP mostly is on packagist these days, there are still some older packages on the pear registry. I think is something we should just consider as part of our implementation design but asking to get more thoughts.

I think the next steps here are to better consolidate our data fetch for library cards and the comparison table. We'll also need to extend the comparison table for each of these different permutations so people can get the best possible apples-to-apples comparisons that exist. It looks like most of the implementation is in these two key locations:

• https://github.com/thisdot/framework.dev/blob/main/packages/system/src/components/cards/library-card.tsx#L35-L82
• https://github.com/thisdot/framework.dev/blob/main/packages/system/src/components/comparison-table/index.tsx#L60-L92

For the Library cards, I'm thinking we'll need to create a custom badging component and for the tables, the data fetch function needs to accept an input to decide which data fetching strategy to execute based on some input.

The conversation is "what is that input value?" Should it be the package manager? Should it be the language or language mode?

[dariodjuric]

Do any of these other package managers provide a badging system like NPM? That would be ideal for display purposes otherwise we need to make generic badges for these stats.

Yes! Most of them are available as badges through shields.io.

PHP

• package.downloads.monthly => Yes
• package.github_stars =>Yes
• package.favers => No
• package.dependents => No
• package.suggesters => No

Ruby

• Daily ranking => Yes
• Daily downloads => Yes
• GitHub stars => Yes

Python

• Last month downloads => Yes
• Classifiers / Development Status => Yes
• Classifiers / Environment => No
• GitHub stars => Yes

Should we worry about alternative package managers?

Good question. I believe in all of the above cases, one package manager is very dominant, so I think we should focus on that one, but that's just my opinion. Even in the PHP case, PEAR is being deprecated in favor of Composer / Packagist. 🤷

[dustinsgoodman]

@dariodjuric If you can work with @Atrophius and generate the follow-up items to add these new features, I'd love that. This could be something you work on or get others involved with.