[DISCUSS] What constitutes a new plugin being a core plugin

[kavilla]

Right now, core plugins for OpenSearch Dashboards means plugins that exist in this repo and "external" plugins are the plugins that exist outside this repo in it's own repo and require an installation step.

As more and more projects get kicked off there appears to be an increasing amount of desire to develop new plugins and place them within the OpenSearch Dashboards repo as a core plugin. Or take existing functionality of an external plugin and move it into the core repo. However, these desires tend to not have discussion on if it's best to include the plugin as a core plugin or not before design. But perhaps it's easier to have a definition what makes a core plugin in OpenSearch Dashboards or not so developers can keep it mind prior to design, because as of right now our current un-official definition is as follows:

If the corresponding OpenSearch plugin does not support the OpenSearch Dashboards without any external plugins then that plugin cannot be apart of core. Regardless if we add custom logic with the OpenSearch Dashboards repo.

But is that enough? From my understanding, OpenSearch's definition of core plugin is any plugin that directly is involved with searching. If the plugin is does not directly involving searching the OpenSearch maintainers will not add that to their repo. OpenSearch Dashboards should have a better definition.

There are pros and cons for either way and I understand developing a plugin as a core plugin will get rid of upfront administrative processes and the end user does not have to install a plugin. But the maintainership becomes extremely blurry as now the maintainers will need to support the new plugin. Also, the new implementation will be highly scrutinized by maintainers prior to merging which might not align with the developers timelines. It's incredibly easy to build an external plugin that extends functionality from core and added to the release pipeline and any processes by maintainers within this repos won't impact developers. It is also important to note, making a plugin a core plugin is an one-way door meaning if it's built in core from the start and released, we shouldn't and most likely will not remove that plugin into it's own external repo as this will have a negative impact downstream. Whereas if a plugin is built as external, and if there is enough metrics and conditions met to say that it should be a core plugin then we can easily migrate the plugin into our repo with a positive impact downstream.

[ashwin-pc]

I think we should also keep in mind the current architecture before making this decision. I'm a strong proponent of making core as light as possible. In fact if core OSD could just be a shell only capable of loading a welcome screen and every feature it provides on top of that be a plugin, i'd love that approach. But that would mean ripping up and rethinking a lot of the current architecture.

To do so without ripping out a lot of things, I propose that we that make every core plugin an external plugin. If thats the case, plugins can depend on other plugins and similar to how npm and yarn resolve dependencies, we can resolve and install all the required plugins while keeping them out of OSD core, keeping the core repo light weight.

[AMoo-Miki]

I love the idea Ashwin is proposing and in fact, came to say exactly the same thing. I would also not hold back on gutting status quo; I propose extending Ashwin's proposal to:

1. Stop entertaining anything new to core: every plugin from this moment onward will be a regular plugin.
2. Make plugin discovery and install be as easy as a couple of clicks: this will not only install the plugins, but will also identify and install any dependency that is not already installed, the mechanism of which is out of scope of this discussion.
3. Pull out anything that is currently called air-quotes core air-quotes and make them regular plugins: nothing will be shipped along with a Dashboards release and the plugins get installed during the FTUE of an admin.

[mihirsoni]

I do agree with @ashwin-pc , IMO Core should be providing following capability fundamentally.

• Chrome for and initial UI
• AuthN and AuthZ.
• Manage metadata and expose interface for plugins to manage their own data.
• Provide plugin framework.
• Dashboards administrative operations (Seamless upgrades / updates).
  Ideally all the application on top could be outside of the core.

[joshuarrrr]

I came here to add to the discussion, but I mostly just have agreement, particularly with @AMoo-Miki's general approach.

[kgcreative]

I'm finding myself nodding along to both @mihirsoni and @AMoo-Miki's points. I think we need to double down on core being the platform that enables features and functionality to be composable, interoperable, extensible, and secure. A huge area of opportunity is in investing in both developer experience (helping plugin developers know how to leverage and interact with the system) and administration (helping cluster administrators configure, add, remove and manage plugins without having to dive deep into the file system and configuration files in order to add plugins and functionality)