feat: Add vim docs & generators

[tjdevries]

No description provided.

[Conni2461]

Do you need help documenting stuff?
Do we want to generate the doc file with CI for Pull Requests similar to how nvim-lspconfig regenerates their README.md for PullRequests?

[kkharji]

Well done @tjdevries

[smithbm2316]

I will take a look into working on helping to set this up in the coming week. Would love to be able to have telescope help docs in Telescope!

[tjdevries]

@Conni2461 do you think we can just merge this in so we can do the rest of the things one file at a time? It seems fine and safe to do.

[tjdevries]

I can add some docs about how to generate before merging

[Conni2461]

Sure we can merge that. :)

I added a CI job that will generate the documentation on push and will only push a new commit if there are changes to the documentation. That way not everyone that is developing a cool new thing needs to remember that he also have to install that plugin, configure it and generate the documentation. For everyone that is still wondering how to do it here are basically the instructions 84c9c95#diff-b332355fbe62a7a35ec05e424ec8ebb9ecf212e7f0c388cecfe022e1c2980811R33-R53 😆 You also have to remove the parser in nvim-treesitter and the queries otherwise there will be errors.

On the other hand if someone wanna fix a typo and just updates the doc, that person will have a bad time because CI will just revert everything. But that way its actually easier to understand that people shouldn't touch doc/telescope.txt :)

[smithbm2316]

I've taken a look through the file changes in this PR, and I think I have the general idea of what is happening, but to be honest I don't know a ton about CI or treesitter. I'm a bit confused as to what it is that was needed for me to do to help get the help menu docs set up for telescope. Is it going and documenting the different functions in Telescope with docstrings, once this PR is merged? I'll try to remember to ask TJ on stream about this, maybe there's just something I'm missing which is why it's not clicking as to how I can help here.

[Conni2461]

Maybe i should break it down a little bit better for the future as well. And maybe we can document that somewhere. Maybe we should introduce a CONTRIBUTING.md. Never mind.

Idea

is generating docs based on the tree sitter syntax tree. TJ wrote a grammar that includes the documentation in this syntax tree so we can do take this function header documentation and transform it into vim documentation. All documentation will be exported that is part of the returning module. So example:

local m = {}

--- Test Header
--@return 1: Returns always 1
function m.a()
  return 1
end

return m

Something like this.

What is missing?

The docgen has some problems on which people can work. This would happen in https://github.com/tjdevries/tree-sitter-lua and documentation of some modules here.
I would suggest we are documenting lua/telescope/builtin/init.lua rather than the files itself. We can use that init.lua file as "header" file, so we are not cluttering the other files.

How to do it:

The easy way would be after this PR is merged

• write some docs
• commit and push
• wait a minute until the CI generates a new commit with the changes
• Look at this commit and the changes
• And apply new changes with git commit --amend and git push --force to remove that github ci commit again.
• Repeat until done

The other option would be setting up https://github.com/tjdevries/tree-sitter-lua

• Install Treesitter, either with package manager or with github release
• Install plugin as usual
• cd to plugin
• mkdir -p build parser Sadly does doesn't exist 😆
• make build_parser
• ln -s ../build/parser.so parser/lua.so We need the so in parser/ so it gets picked up by neovim. Either copy or symbolic link
• Make sure that nvim-treesitter lua parser is not installed and also delete the lua queries in that repository. queries/lua/*. If you are not doing that you will have a bad time
• cd into this project
• Write doc
• Run make docgen
• Repeat last two steps

Okay i hope i haven't missed something and this answers your questions. If not or if something doesn't work correctly feel free to leave them here.

[smithbm2316]

@Conni2461 sweet! Thank you so much for the thorough explanation, and now from chatting on stream I see how I can contribute here. I'll definitely plan on reading through the telescope codebase in the coming weeks to see what I can help document. I'm new to Lua, but I'm also trying to write my own Lua Neovim plugin at the moment, so I'm sure getting to know the Telescope codebase better and helping to document it will help me in my own pursuit of making my own Neovim plugin with Lua!

[Conni2461]

You wanna know what it does :)

[tjdevries]

@smithbm2316 I'm pretty close to getting this merged, then you can start adding documentation to things like builtins would be super helpful. If there's anything else you're interested in, let me know, we can start there (like layout, previewers, etc.)

[smithbm2316]

@smithbm2316 I'm pretty close to getting this merged, then you can start adding documentation to things like builtins would be super helpful. If there's anything else you're interested in, let me know, we can start there (like layout, previewers, etc.)

@tjdevries Great, thanks for the update! I will definitely start there. I've actually already been poking around in the builtin directory a bit, I added PR #548 last night with a small feature that I've been wanting (which also coincidentally would resolve issue #438), so I'm starting to get my bearings around the builtins a bit 😃. Looking forward to helping with the docs in the near future!

[tjdevries]

OK, I'm merging. This is a good start. Anyone can update docs how they want after this (and we move some of the readme into the help docs now, and just say "see :help telescope.blah" or whatever. This will at least unblock people from doing new docs.