chore: add /generator path

[magicmatatjahu]

Description

Add /generator path:

• in first iteration generate only two official templates, html and markdown
• add middleware for validation template parameters

Initialize /src structure:

• separate logic: controllers (logic for route and their initialization), services (business logic and helpers)
• add global middleware to catch all ProblemException error
• add global middleware to validate AsyncAPI document being sent
• add global middleware to validate request body based on operation's schema from OpenAPI Document
• add unit tests

OpenAPI document:

• change name of allowed templates, before html and markdown, now @asyncapi/html-template and @asyncapi/markdown-template
• add required fields to the Problem schema

Some insights:

I think that we should go with names of paths with verbs like in our CLI, so /generator should be /generate, in future /validate, /convert, /bundle etc. I know that we decided go with REST patterns, but it doesn't work, really, I always wrote (in tests and source code) /generate path rather than /generator so it's more natural.

cc: @smoya @jonaslagoni @fmvilas @derberg
If you want share your opinions @BOLT04 I would be grateful, but if you don't have time, it's okay :)

[jonaslagoni]

A few comments 🙂 Looking good 👍

[jonaslagoni]

LGTM 👍

[fmvilas]

Yeah, I'd not go with REST design for this API. Pure RPC seems a better fit IMHO. All methods will be POST and paths will always be verbs. The rest of the info can be provided in the request payload. Like you said, @magicmatatjahu, just like a CLI.

[fmvilas]

I'd just change the path to /generate. Other than that, looks good to me.

[smoya]

How this is gonna work? What are the credentials? Are they gonna be based on a cookie? I'm confused.

[magicmatatjahu]

It's used for parser options and it's exactly options for json-schema-ref-parser. More info here :) https://github.com/APIDevTools/json-schema-ref-parser/blob/main/docs/options.md#resolve-options In this same way we do in old playground :)

[smoya]

I still think we should move into generating in memory, writing directly to the zip stream by using (If I'm correct) append archiver method: https://www.archiverjs.com/docs/archiver#append.

The whole process will be generated on the fly, way faster, and theoretically, since you piped the stream to the response object, data should be flushed on each write, isn't? Meaning you don't hold everything in memory but just a small buffer since everything is drained though the response (sent back to the client).

[magicmatatjahu]

@smoya Maybe I'm not understanding it correctly, but how to do that if generator uses under the hood filesystem api to "render" templates? For that I passed directory destination for output.

[smoya]

I just want to drop a comment regarding the whole design pattern used on this app. We are currently following some sort of MVC pattern, which it is enough for simple applications IMHO (like this one today).

However, I can detect and I would say even foresee that the logic on this API will be very similar, in part, to what the CLI does nowadays.
What about moving into a more Hexagonal Architecture pattern, where we can reuse most of the logic of both apps but the only (or mostly) thing that changes are ports and adapters?
Maybe worth discussing this in a new issue or discussion.

[smoya]

I just want to drop a comment regarding the whole design pattern used on this app. We are currently following some sort of MVC pattern, which it is enough for simple applications IMHO (like this one today).

However, I can detect and I would say even foresee that the logic on this API will be very similar, in part, to what the CLI does nowadays.
What about moving into a more Hexagonal Architecture pattern, where we can reuse most of the logic of both apps but the only (or mostly) thing that changes are ports and adapters?
Maybe worth discussing this in a new issue or discussion.

[BOLT04]

I just want to drop a comment regarding the whole design pattern used on this app. We are currently following some sort of MVC pattern, which it is enough for simple applications IMHO (like this one today).

However, I can detect and I would say even foresee that the logic on this API will be very similar, in part, to what the CLI does nowadays. What about moving into a more Hexagonal Architecture pattern, where we can reuse most of the logic of both apps but the only (or mostly) thing that changes are ports and adapters? Maybe worth discussing this in a new issue or discussion.

@smoya I totally agree on moving to another architecture if you foresee most logic will be reusable on this API and the CLI 👍.
I'm more familiar with Clean Architecture which I believe is inspired by the hexagonal architecture and others.
Both have the same ideas IMO, we would have a different TS module with the core of the "system". This core module will have that reusable logic you mentioned, and is framework independent so that we can use it in this HTTP API with Express.js or with oclif in the CLI tool. Not just these two but other apps that use different "ports" that we can adapt to the core's interface 🙂.

Do you think it's worth chasing a move towards this architecture now, or keep it simple until we find the need to change the architecture (I'm kinda debating whether to follow the KISS principle or do more upfront design 😅)?

Could you share briefly what sort of areas you see being reused? I'm asking because I had a look at this PR and the CLI repo, but couldn't find the type of logic that this API would use (I mostly looked at commands and models).

[magicmatatjahu]

@smoya @BOLT04 Thanks for review and for feedback!

Honestly, about hexagonal architecture, I don't think so that it will be good idea. CLI (as @BOLT04 noted) has models and command, server-api will have controllers/routes (and some services). Even if we wanna create reusable parts (like services) we should see that for CLI/Server-API given service can be significantly different for two given environments. For example the generator - on the server side we have to generate in temporary folders and send the zip, in the CLI we generate in the folder specified by the user. I can say that currently we are using quasi-hexagonal pattern due to the fact that every tool in our organization is separate library and can be easily integrated into every environment. Why make life difficult for yourself, make wrappers/adapters in one, shared place and then again in another place, whether it is a server or CLI? 🤷🏼

[smoya]

@smoya @BOLT04 Thanks for review and for feedback!

Honestly, about hexagonal architecture, I don't think so that it will be good idea. CLI (as @BOLT04 noted) has models and command, server-api will have controllers/routes (and some services). Even if we wanna create reusable parts (like services) we should see that for CLI/Server-API given service can be significantly different for two given environments. For example the generator - on the server side we have to generate in temporary folders and send the zip, in the CLI we generate in the folder specified by the user. I can say that currently we are using quasi-hexagonal pattern due to the fact that every tool in our organization is separate library and can be easily integrated into every environment. Why make life difficult for yourself, make wrappers/adapters in one, shared place and then again in another place, whether it is a server or CLI? 🤷🏼

Maybe its time to change the generator :).

Anyway, I will move this into separate issues so we are not completely blocking this pr.

[smoya]

There are a couple of concerns I will like to resolve asap. However, im gonna approve this PR so we can discuss those deeply and move forward without blocking.

🚀 Nice job @magicmatatjahu !!!💯

[BOLT04]

LGTM 👍

[BOLT04]

Anyway, I will move this into separate issues so we are not completely blocking this pr.

@smoya could you add the link for the issue afterward here? I'd like to follow that discussion 🙂

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
15 Code Smells

No Coverage information
0.8% Duplication

[magicmatatjahu]

I see some code smells from SonarCloud. I will merge that PR with chore: to not trigger release, because we don't have yet integrated needed packages to release automation, I will also fix that issues in the next PR and we can release 0.1.0 :)

[smoya]

@smoya @BOLT04 Thanks for review and for feedback!

Honestly, about hexagonal architecture, I don't think so that it will be good idea. CLI (as @BOLT04 noted) has models and command, server-api will have controllers/routes (and some services). Even if we wanna create reusable parts (like services) we should see that for CLI/Server-API given service can be significantly different for two given environments. For example the generator - on the server side we have to generate in temporary folders and send the zip, in the CLI we generate in the folder specified by the user. I can say that currently we are using quasi-hexagonal pattern due to the fact that every tool in our organization is separate library and can be easily integrated into every environment. Why make life difficult for yourself, make wrappers/adapters in one, shared place and then again in another place, whether it is a server or CLI? 🤷🏼

Maybe its time to change the generator :).

Anyway, I will move this into separate issues so we are not completely blocking this pr.

@magicmatatjahu @BOLT04 I created a discussion about this topic with a more expanded context. You might want to look at it asyncapi/community#212