Automatically search for grammar on GitHub when none is configured yet

[mickaelistria]

GitHub textmate organization is actually a repo of TextMate grammars. When a file is opened with Generic Editor and expects TM4E to participate but tm4e has no grammar, it should open a dialog to user suggesting to search for grammars on GitHub and allow to fetch them. A query like "<string>cpp</string>" user:textmate repo:*.tmbundle extension:plist (in case of .cpp) can for example show interesting results.

[angelozerr]

I love this great idea! But it means that we must use git protocol to download it?

[mickaelistria]

GitHub does allow to access raw content of repos through http: https://raw.githubusercontent.com/textmate/c.tmbundle/master/Syntaxes/C.plist

[angelozerr]

Yes you are right, but how we can map a textmate grammar with this URL? I mean genericeditor opens a file with extends .c, how we can discover the well URL https://raw.githubusercontent.com/textmate/c.tmbundle/master/Syntaxes/C.plist? Perhaps we should have a textmate serach engine which does that?

[angelozerr]

An idea is to have a server like https://tmsearch.com/ which does the same thing for npm https://npmsearch.com/ (to find package)

[angelozerr]

@ainslec perhaps this textmate search engine could b ean issue for https://eeyo.io/iro/ ? What do you think about that?

[mickaelistria]

but how we can map a textmate grammar with this URL?

We don't map it, we send the search request to GitHub and tell user "We found this grammar from GitHub, do you want to use it?". No additional registry involved, GitHub is the registry and search engine.

[angelozerr]

"We found this grammar from GitHub, do you want to use it?"

Sorry I have not understood your idea?

An idea is that we could use teh APi Search of github https://developer.github.com/v3/search/

For instance:

• we can retrieve list of github repositories with https://api.github.com/search/repositories?q=tmBundle

{
  "total_count": 1673,
  "incomplete_results": false,
  "items": [
    {
      "id": 793799,
      "name": "SCSS.tmbundle",
      "full_name": "MarioRicalde/SCSS.tmbundle",
      "owner": {
        "login": "MarioRicalde",
        "id": 125900,
        "avatar_url": "https://avatars0.githubusercontent.com/u/125900?v=4",
        "gravatar_id": "",
        "url": "https://api.github.com/users/MarioRicalde",
....

• after we can retrieve the URL like this https://api.github.com/search/code?q=repo:MarioRicalde/SCSS.tmbundle+extension:tmLanguage

{
  "total_count": 1,
  "incomplete_results": false,
  "items": [
    {
      "name": "SCSS.tmLanguage",
      "path": "Syntaxes/SCSS.tmLanguage",

• we can build the raw URL by using ""full_name": "MarioRicalde/SCSS.tmbundle" and "path": "Syntaxes/SCSS.tmLanguage", https://raw.githubusercontent.com/MarioRicalde/SCSS.tmbundle/master/Syntaxes/SCSS.tmLanguage

[ainslec]

@angelozerr

perhaps this textmate search engine could b ean issue for https://eeyo.io/iro/ ? What do you think about that?

What kind of feature do you want it to be? I understand the concept of a .tmLanguage registry, but what role do you think Iro should play? Should it add an option to publish to a central registry, or provide a registry itself? Just trying to understand your comment.

[angelozerr]

What kind of feature do you want it to be?

Just having a textmate search engine like search textmate grammar .tmLanguage by file extension which links to grammar (not host the grammar, just give a link that we can download it)

[ainslec]

Problem is that Iro is a static web page, nothing more. And it doesn't store anything on the serverside (because there is no serverside).

Seems like the problem is not a tech problem, but a hosting problem. Maybe the Eclipse project can host the search engine?

[angelozerr]

Seems like the problem is not a tech problem, but a hosting problem. Maybe the Eclipse project can host the search engine?

I think we can consume the Github API Search for that even if it's not perfect, but IMHO I think the better clean solution is to have a search engine like we have for npm.

[mickaelistria]

Creating and hosting a search engine is not an affordable effort for this project (and is not really the topic of this issue initially) and I'm confident that sending good queries to GitHub search service would work well for the vast majority of cases.

[angelozerr]

A query like "cpp" user:textmate repo:*.tmbundle extension:plist (in case of .cpp) can for example show interesting results.

Could you give me the right github URL which retrieves this grammar please?

[mickaelistria]

I'm going submarine and doing only bugfix, releng or consolidation of existing features for Photon. I'll come back to this topic in a few weeks as discovery (of Docker based LS, or TM grammars) is likely to become a hot topic for me.

[angelozerr]

Ok I wanted to start something, but if you have the intetion to do it, I let you implement this feature.