Add support for gRPC

[jwulf]

Is it possible to write a gRPC client application in Deno?

[benjamingr]

Sure, what do you mean?

[ry]

There is no gRPC module yet.

[tuddman]

would this be the way to go? https://github.com/stepancheg/grpc-rust

[sleipnir]

Hello @tuddman , I understand that Deno is written over Rust but I think the question is about how I use gRPC in Deno, how do I write a gRPC client or server code in Deno? It's possible? Is there a library ready to use? If not, how to use the link you sent to provide a Javascript / Typescript api at the user level?

[ajile-in]

Looking forward to the possibility of using gRPC with Deno.

[sleipnir]

I believe that due to the popularity of gRPC it would be great to see support in the Deno standard library, like any other form of IO

[theoutlander]

I would like to help with this integration, but I'm wondering if the best option is integration with grpc-rust or writing a typescript library? What about porting something like https://github.com/grpc/grpc-web/tree/master/packages/grpc-web as a typescript lib?

[kitsonk]

IMO, this is something that should start external to Deno/std and move into std. It should either be a native module or Web Assembly.

[badsyntax]

It should be noted the current advised approach of using gRPC in Node.js is to use the pure JavaScript module which uses Node.js's built-in modules (http2) instead of a native module built in c++. The module is built in TypeScript and perhaps can be used for inspiration for something in Deno, assuming http2 support is provided by Deno at some point. cc @murgatroid99

[murgatroid99]

Hi. I'm the author of the existing gRPC libraries for Node.js. I think that porting @grpc/grpc-js to Deno might be the way to go, but that would depend on how close Deno's eventual HTTP/2 implementation is to Node's http2 module, semantically. Other built in modules may also be a concern, particularly tls.

I would not suggest porting grpc-web. That library implements a different protocol to account for the restrictions of browser APIs. It needs a proxy to talk to regular gRPC servers.

[sleipnir]

I aggre about grpc-web maybe this should be another Deno module

[yuri4n]

@murgatroid99 I have a doubt, how is gRPC-node generating code from Proto right now?

[murgatroid99]

There are two major methods: one is to use the @grpc/proto-loader package to load .proto files and generate classes and object at runtime. That uses Protobuf.js internally.

The alternative is to use the protoc tool distributed with a plugin in the grpc-tools package to generate JavaScript files that depend on the google-protobuf package.

I want to note that neither of these is strictly necessary in general to use grpc. The gRPC protocol (and the library) can handle any kinds of serializable messages.

[yuri4n]

Ok, let me understand a little bit. So first we need to implement something like grpc-deno, which should have support for any kind of serializable message. Then adapting (or build a new one) the tooling for generating grpc code that uses grpc-deno as a reference, from proto (or any other) to TypeScript/Javascript.

[murgatroid99]

That looks about right.

One thing I want to note is that there is a kind of generic interchange format for defining a collection of services that grpc.loadPackageDefinition accepts and @grpc/proto-loader generates, and there is also an option to have grpc-tools generate it. That interchange format is defined as PackageDefinition in this document. I think it would be good if any implementation for Deno can load the same kind of objects.

Ideally I think gRPC for Deno should have the same API and functionality as the existing @grpc/grpc-js, but I recognize that that may not be feasible.

[yuri4n]

@murgatroid99 there is an article or a document explaining how grpc-node (or other C-grpc-based libraries) uses the core-grpc implementation?

[yuri4n]

Another thing @murgatroid99, What do you think about building a grpc-deno on top of tonic (pure Rust implementation of grpc).

[murgatroid99]

I'm not aware of such a public document about using the gRPC core library. We did most of that work before we started doing our design work publicly. The simple answer, though, is that we defined a public API for JavaScript users (in this case), and then an intermediate API, that the implementation of the public API could consume and that we could reasonably implement in C++ using the core API. That intermediate API is considered non-public and not guaranteed to be stable. If you look at the grpc package directories, the ext/ directory contains the implementation of that intermediate API and the src/ directory contains the implementation of the public API.

The Node library in particular is unusual in that the gRPC core library has a libuv implementation of the low-level network interaction that was created specifically for the Node library, which can be enabled using a compiler flag. So the Node gRPC library enables that to simplify its interaction with the event loop.

I think it's important to mention that the native addon was a maintainability nightmare, to the point that we decided to reimplement the library purely in TypeScript as @grpc/grpc-js and we are phasing out the use of the native addon-based library in favor of the pure TypeScript implementation.

Regarding implementing this on top of tonic, I think the important question is how close the desired JavaScript API is to the tonic API, semantically. In addition, how much extra work will it be to maintain the Deno equivalent of a native addon using a Rust library? And how does the total implementation work there compare with modifying @grpc/grpc-js for use with Deno, once the necessary standard libraries are available.

[yuri4n]

I have taken a look at your pure TS implementation of grpc, and you are using only a single node dependency, so a deno library could be hard based on that implementation right? Based on what you said about the core library could be a nightmare in the long term, even with a Rust implementation.

[theoutlander]

I like the typescript based approach. Maybe we need to implement http/2 first, then it is mostly straightforward.

…

On Sat, Aug 15, 2020 at 11:36 AM yurianxdev ***@***.***> wrote: I have taken a look at your pure TS implementation of grpc, and you are using only a single node dependency, so a deno library couldn't be hard based on that implementation? Based on what you said about the core library could be a nightmare in the long term, even with a Rust implementation. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3326 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFW4HFVFN36VLGRWRRF75DSA3ITXANCNFSM4JMRSKMA> .

[yuri4n]

There is no progress on HTTP2 (#3995) so if we want to follow the pure TS approach we need to implement it. That will kill two birds with one shot, I like it.
@murgatroid99 what do you think about a pure TS implementation based on yours?

[kitsonk]

HTTP2 will eventually land.

Again, this needs to start as something seperate from std or the Deno cli.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[jd1378]

any news on this ?

[anonrig]

Hi! I'm a maintainer from mali (GRPC microservice framework) and wanted to stay in the loop. We're currently investigating supporting Deno in Mali. If you're interested in this support, please send us a message from malijs/mali#241

[brunoluiz]

Seems Deno 1.9, released today, implements HTTP 2.0 🎉

https://deno.com/blog/v1.9#native-http%2F2-web-server

[ghost]

Support for gRPC in the Deno CLI will be an extremely nice feature, thank you for choosing to do so!
I think that supporting the Connect protocol (with gRPC and gRPC-Web) would be interesting.
Is it possible to contribute to the development of gRPC for Deno?

[disjukr]

FYI, I posted a runtime API proposal for gRPC support in the discussions. #17308

[donoso-eth]

Hey, what is the status here? Can we expect support for gRPC ?

[satyarohith]

Hi @donoso-eth. We are actively working to support https://www.npmjs.com/package/@grpc/grpc-js in Deno. Right now, the following official client examples work if you use es modules (here is a branch with updated examples)

• helloworld
• cancellation
• interceptors
• deadline
• metadata
• keepalive

The next steps are to get the error_handling example working and popular packages that rely on gRPC.

Note that server examples don't work yet and we will focus on them once we have the clients working.

[hronro]

It's very cool to know grpc-js will be available on Deno soon, I would also like to know what the performance it would be.

Since Deno already has a very performant built-in HTTP server, I would expect Deno will eventually have a gRPC implementation at the same performance level. I also wonder if it's possible to achieve that performance via pure JavaScript, or if some parts must be written in Rust.

[birkskyum]

So the blog post make it look like everything is working. Is that the case? Anyone how has a repro readily available to try this out?

https://deno.com/blog/v1.44#grpc-connections-now-supported

Related

• npm:@grpc/grpc-js support for server-side #23714
• gRPC / node:http2 #23246

[satyarohith]

@birkskyum we have support for all of the official gRPC client side examples and we also tested google packages like vision that rely on gRPC client APIs and they are working. We don't support the server side yet.

Edit:

Re: example

The example from the blog post should work with GCP Application Default Credentials setup and an image on the local disk to test.

If you do find any issue, please report and we will fix them.

[xiaojiudev]

@rnbguy @satyarohith I am using grpc_basic, but it's getting an error, and the author of that library is no longer providing support. For the grpc_basic example in the server, it needs Deno to listen on a port, and the grpc_server will handle that port (Deno.TCPConn), as shown in the image below:

Now, as you say, Deno v1.44 now support grpc/grpcjs. so I want to implement grpc/grpc-js, but I have a small issue. The library needs bindAsync directly on the port, and there's no longer a need for using Deno.listen. However, I want to know if it's possible to use Deno.listen instead?

[rnbguy]

hey @xiaojiudev, I am not really part of the Deno team. But AFAIK, the current Deno @grpc/grpc-js support is only for gRPC clients.

[niteshbalusu11]

I am trying to get a grpc client working in deno. I used protobufjs-cli to generate a JS and d.ts file.
When importing the file and running in deno I am getting the following error:

error: The module's source code could not be parsed: 'import', and 'export' cannot be used outside of module code at file:///Users/nitesh/Documents/dunder-lsp-lsps/src/proto.mjs:19:5

      export const lnrpc = $root.lnrpc = (() => {

I tried changing the file to .mjs as well.

Also inside the generated JS file, getting this where it says export const lnrpc

Modifiers cannot appear here.

Generated JS file looks something like this:

/*eslint-disable block-scoped-var, id-length, no-control-regex, no-magic-numbers, no-prototype-builtins, no-redeclare, no-shadow, no-var, sort-vars*/
(function(global, factory) { /* global define, require, module */

    /* AMD */ if (typeof define === 'function' && define.amd)
        define(["protobufjs/minimal"], factory);

    /* CommonJS */ else if (typeof require === 'function' && typeof module === 'object' && module && module.exports)
        module.exports = factory(require("protobufjs/minimal"));

})(this, function($protobuf) {
    "use strict";

    // Common aliases
    const $Reader = $protobuf.Reader, $Writer = $protobuf.Writer, $util = $protobuf.util;
    
    // Exported root namespace
    const $root = $protobuf.roots["default"] || ($protobuf.roots["default"] = {});
    
    export const lnrpc = $root.lnrpc = (() => {
    
        /**
         * Namespace lnrpc.
         * @exports lnrpc
         * @namespace
         */
        const lnrpc = {};
    
        lnrpc.Lightning = (function() {
    
            /**
             * Constructs a new Lightning service.
             * @memberof lnrpc
             * @classdesc Represents a Lightning
             * @extends $protobuf.rpc.Service
             * @constructor
             * @param {$protobuf.RPCImpl} rpcImpl RPC implementation
             * @param {boolean} [requestDelimited=false] Whether requests are length-delimited
             * @param {boolean} [responseDelimited=false] Whether responses are length-delimited
             */
            function Lightning(rpcImpl, requestDelimited, responseDelimited) {
                $protobuf.rpc.Service.call(this, rpcImpl, requestDelimited, responseDelimited);
            }
    
    ```
    
    

[garethj2]

I'm running Deno 1.44.4 and have a client working. Unfortunately everything seems to stop working after 1 minute 40 seconds. I've run exactly the same code using NodeJS and there is no problem.

My code simply opens a stream and listens for events.

Unfortunately I don't have any errors I can report from the output.

[satyarohith]

@garethj2 the issue you reported is fixed in #24576

[butadpj]

I just submitted a new issue - #26436 that seems related to Deno's lack of support for server-side gRPC.

[satyarohith]

Closing this thread as the original issue is about gRPC client applications which are now supported. Please open new issues for bugs related to clients. And support for server side is being tracked at #23714