Firestore Client Does Not Work Behind a Proxy #493

hiranya911 · 2018-12-15T20:13:54Z

Environment details

OS: Ubuntu 16.04
Node.js version: v8.11.1
npm version: 5.6.0
@google-cloud/firestore version: 0.19.0

Steps to reproduce

Trying to access Firestore through a simple HTTP proxy. According to the instructions in this article, Node.js GRPC supports specifying a proxy via an environment variable.

Set the HTTPS_PROXY environment variable
Try to access any Firestore API

GRPC debug log

D1215 12:10:04.861744405   17298 dns_resolver.cc:338]        Using native dns resolver
D1215 12:10:04.937711172   17298 dns_resolver.cc:279]        Start resolving.
I1215 12:10:04.964020153   17298 ssl_transport_security.cc:174] OpenSSL callback has already been set.
I1215 12:10:05.012894812   17298 handshaker.cc:141]          handshake_manager 0x34c5280: adding handshaker http_connect [0x34c53d0] at index 0
I1215 12:10:05.013020674   17298 handshaker.cc:141]          handshake_manager 0x34c5280: adding handshaker security [0x34df1a0] at index 1
I1215 12:10:05.013071119   17298 handshaker.cc:212]          handshake_manager 0x34c5280: error="No Error" shutdown=0 index=0, args={endpoint=0x3341fb0, args=0x34c2c00 {size=9: grpc.primary_user_agent=grpc-node/1.16.1, grpc.client_channel_factory=0x7fe7bf9e6520, grpc.channel_credentials=0x2d3cfc0, grpc.server_uri=dns:///firestore.googleapis.com:443, grpc.channelz_channel_node_creation_func=0x7fe7bf772e50, grpc.default_authority=firestore.googleapis.com:443, grpc.http2_scheme=https, grpc.security_connector=0x2e8f690, grpc.subchannel_address=ipv6:[2607:f8b0:4005:805::200a]:443}, read_buffer=0x34df630 (length=0), exit_early=0}
I1215 12:10:05.013076542   17298 handshaker.cc:258]          handshake_manager 0x34c5280: calling handshaker http_connect [0x34c53d0] at index 0
I1215 12:10:05.013104710   17298 handshaker.cc:212]          handshake_manager 0x34c5280: error="No Error" shutdown=0 index=1, args={endpoint=0x3341fb0, args=0x34c2c00 {size=9: grpc.primary_user_agent=grpc-node/1.16.1, grpc.client_channel_factory=0x7fe7bf9e6520, grpc.channel_credentials=0x2d3cfc0, grpc.server_uri=dns:///firestore.googleapis.com:443, grpc.channelz_channel_node_creation_func=0x7fe7bf772e50, grpc.default_authority=firestore.googleapis.com:443, grpc.http2_scheme=https, grpc.security_connector=0x2e8f690, grpc.subchannel_address=ipv6:[2607:f8b0:4005:805::200a]:443}, read_buffer=0x34df630 (length=0), exit_early=0}
I1215 12:10:05.013122409   17298 handshaker.cc:258]          handshake_manager 0x34c5280: calling handshaker security [0x34df1a0] at index 1
I1215 12:10:05.070240703   17298 handshaker.cc:212]          handshake_manager 0x34c5280: error="No Error" shutdown=0 index=2, args={endpoint=0x34e02c0, args=0x34e7230 {size=10: grpc.primary_user_agent=grpc-node/1.16.1, grpc.client_channel_factory=0x7fe7bf9e6520, grpc.channel_credentials=0x2d3cfc0, grpc.server_uri=dns:///firestore.googleapis.com:443, grpc.channelz_channel_node_creation_func=0x7fe7bf772e50, grpc.default_authority=firestore.googleapis.com:443, grpc.http2_scheme=https, grpc.security_connector=0x2e8f690, grpc.subchannel_address=ipv6:[2607:f8b0:4005:805::200a]:443, grpc.auth_context=0x34e9150}, read_buffer=0x34df630 (length=0), exit_early=0}
I1215 12:10:05.070254897   17298 handshaker.cc:245]          handshake_manager 0x34c5280: handshaking complete -- scheduling on_handshake_done with error="No Error"
I1215 12:10:05.070331159   17298 subchannel.cc:656]          New connected subchannel at 0x34d4a60 for subchannel 0x2f7a550
Auth error:Error: write EPROTO 140633384175424:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:../deps/openssl/openssl/ssl/s23_clnt.c:827:

Auth error:Error: write EPROTO 140633384175424:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:../deps/openssl/openssl/ssl/s23_clnt.c:827:

Auth error:Error: write EPROTO 140633384175424:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:../deps/openssl/openssl/ssl/s23_clnt.c:827:

Auth error:Error: write EPROTO 140633384175424:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:../deps/openssl/openssl/ssl/s23_clnt.c:827:

Auth error:Error: write EPROTO 140633384175424:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:../deps/openssl/openssl/ssl/s23_clnt.c:827:

Auth error:Error: write EPROTO 140633384175424:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:../deps/openssl/openssl/ssl/s23_clnt.c:827:

Auth error:Error: write EPROTO 140633384175424:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:../deps/openssl/openssl/ssl/s23_clnt.c:827:

Auth error:Error: write EPROTO 140633384175424:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:../deps/openssl/openssl/ssl/s23_clnt.c:827:

Auth error:Error: write EPROTO 140633384175424:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:../deps/openssl/openssl/ssl/s23_clnt.c:827:

Auth error:Error: write EPROTO 140633384175424:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:../deps/openssl/openssl/ssl/s23_clnt.c:827:

Auth error:Error: write EPROTO 140633384175424:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:../deps/openssl/openssl/ssl/s23_clnt.c:827:

Auth error:Error: write EPROTO 140633384175424:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:../deps/openssl/openssl/ssl/s23_clnt.c:827:

Auth error:Error: write EPROTO 140633384175424:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:../deps/openssl/openssl/ssl/s23_clnt.c:827:

Auth error:Error: write EPROTO 140633384175424:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:../deps/openssl/openssl/ssl/s23_clnt.c:827:

Auth error:Error: write EPROTO 140633384175424:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:../deps/openssl/openssl/ssl/s23_clnt.c:827:

(node:17298) UnhandledPromiseRejectionWarning: Error: 14 UNAVAILABLE: Getting metadata from plugin failed with error: write EPROTO 140633384175424:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:../deps/openssl/openssl/ssl/s23_clnt.c:827:

    at Object.exports.createStatusError (/home/hiranya/Projects/firebase/proxy-config/node-example/node_modules/grpc/src/common.js:87:15)
    at ClientReadableStream._emitStatusIfDone (/home/hiranya/Projects/firebase/proxy-config/node-example/node_modules/grpc/src/client.js:235:26)
    at ClientReadableStream._receiveStatus (/home/hiranya/Projects/firebase/proxy-config/node-example/node_modules/grpc/src/client.js:213:8)
    at Object.onReceiveStatus (/home/hiranya/Projects/firebase/proxy-config/node-example/node_modules/grpc/src/client_interceptors.js:1256:15)
    at InterceptingListener._callNext (/home/hiranya/Projects/firebase/proxy-config/node-example/node_modules/grpc/src/client_interceptors.js:564:42)
    at InterceptingListener.onReceiveStatus (/home/hiranya/Projects/firebase/proxy-config/node-example/node_modules/grpc/src/client_interceptors.js:614:8)
    at /home/hiranya/Projects/firebase/proxy-config/node-example/node_modules/grpc/src/client_interceptors.js:1019:24

HTTP proxy log (Squid proxy access log)

1544904335.228      0 172.17.0.1 TAG_NONE/400 4390 NONE error:invalid-request - HIER_NONE/- text/html
1544904337.509      0 172.17.0.1 TAG_NONE/400 4390 NONE error:invalid-request - HIER_NONE/- text/html
1544904342.295      0 172.17.0.1 TAG_NONE/400 4390 NONE error:invalid-request - HIER_NONE/- text/html
1544904342.403      0 172.17.0.1 TAG_NONE/400 4390 NONE error:invalid-request - HIER_NONE/- text/html
1544904605.087      0 172.17.0.1 TAG_NONE/400 4390 NONE error:invalid-request - HIER_NONE/- text/html
1544904607.963      0 172.17.0.1 TAG_NONE/400 4390 NONE error:invalid-request - HIER_NONE/- text/html
1544904612.874      0 172.17.0.1 TAG_NONE/400 4390 NONE error:invalid-request - HIER_NONE/- text/html
1544904612.984      0 172.17.0.1 TAG_NONE/400 4390 NONE error:invalid-request - HIER_NONE/- text/html
1544904615.098      0 172.17.0.1 TAG_NONE/400 4390 NONE error:invalid-request - HIER_NONE/- text/html
1544904619.662      0 172.17.0.1 TAG_NONE/400 4390 NONE error:invalid-request - HIER_NONE/- text/html
1544904619.768      0 172.17.0.1 TAG_NONE/400 4390 NONE error:invalid-request - HIER_NONE/- text/html
1544904622.422      0 172.17.0.1 TAG_NONE/400 4390 NONE error:invalid-request - HIER_NONE/- text/html
1544904626.594      0 172.17.0.1 TAG_NONE/400 4038 ��� 1%84k%12%C2=%14%BD%1A6%D8%CC%C4%9ELz%DA%60%B6%I%8E. - HIER_NONE/- text/html
1544904626.698      0 172.17.0.1 TAG_NONE/400 4390 NONE error:invalid-request - HIER_NONE/- text/html
1544904629.113      0 172.17.0.1 TAG_NONE/400 4390 NONE error:invalid-request - HIER_NONE/- text/html
1544904634.095      0 172.17.0.1 TAG_NONE/400 4390 NONE error:invalid-request - HIER_NONE/- text/html
1544904634.199      0 172.17.0.1 TAG_NONE/400 4390 NONE error:invalid-request - HIER_NONE/- text/html
1544904636.265      0 172.17.0.1 TAG_NONE/400 4390 NONE error:invalid-request - HIER_NONE/- text/html
1544904640.374      0 172.17.0.1 TAG_NONE/400 4390 NONE error:invalid-request - HIER_NONE/- text/html

Note that this problem is unique to the Firestore Node.js client. I've tried the same with Java and Go clients where this use case works fine.

The text was updated successfully, but these errors were encountered:

hiranya911 · 2018-12-15T21:59:40Z

It seems the auth library is at fault here. It's not connecting to the HTTP proxy correctly.

If I just set the HTTP_PROXY variable, I can get Firestore to go through the proxy. But in that case the credentials will not go through the proxy.

D1215 13:57:28.712727343   26718 dns_resolver.cc:338]        Using native dns resolver
D1215 13:57:28.787131284   26718 dns_resolver.cc:279]        Start resolving.
I1215 13:57:28.800943867   26718 ssl_transport_security.cc:174] OpenSSL callback has already been set.
I1215 13:57:28.821523393   26718 handshaker.cc:141]          handshake_manager 0x35d2170: adding handshaker http_connect [0x338d2c0] at index 0
I1215 13:57:28.821620747   26718 handshaker.cc:141]          handshake_manager 0x35d2170: adding handshaker security [0x35d1090] at index 1
I1215 13:57:28.821647440   26718 handshaker.cc:212]          handshake_manager 0x35d2170: error="No Error" shutdown=0 index=0, args={endpoint=0x33cfed0, args=0x3318ca0 {size=10: grpc.primary_user_agent=grpc-node/1.16.1, grpc.client_channel_factory=0x7f406739f520, grpc.channel_credentials=0x33b2480, grpc.server_uri=dns:///firestore.googleapis.com:443, grpc.channelz_channel_node_creation_func=0x7f406712be50, grpc.http_connect_server=firestore.googleapis.com:443, grpc.default_authority=firestore.googleapis.com:443, grpc.http2_scheme=https, grpc.security_connector=0x34ca5e0, grpc.subchannel_address=ipv4:127.0.0.1:3128}, read_buffer=0x338e500 (length=0), exit_early=0}
I1215 13:57:28.821656912   26718 handshaker.cc:258]          handshake_manager 0x35d2170: calling handshaker http_connect [0x338d2c0] at index 0
I1215 13:57:28.821662952   26718 http_connect_handshaker.cc:300] Connecting to server firestore.googleapis.com:443 via HTTP proxy ipv4:127.0.0.1:3128
I1215 13:57:28.851203756   26718 handshaker.cc:212]          handshake_manager 0x35d2170: error="No Error" shutdown=0 index=1, args={endpoint=0x33cfed0, args=0x3318ca0 {size=10: grpc.primary_user_agent=grpc-node/1.16.1, grpc.client_channel_factory=0x7f406739f520, grpc.channel_credentials=0x33b2480, grpc.server_uri=dns:///firestore.googleapis.com:443, grpc.channelz_channel_node_creation_func=0x7f406712be50, grpc.http_connect_server=firestore.googleapis.com:443, grpc.default_authority=firestore.googleapis.com:443, grpc.http2_scheme=https, grpc.security_connector=0x34ca5e0, grpc.subchannel_address=ipv4:127.0.0.1:3128}, read_buffer=0x338e500 (length=0), exit_early=0}
I1215 13:57:28.851230553   26718 handshaker.cc:258]          handshake_manager 0x35d2170: calling handshaker security [0x35d1090] at index 1
I1215 13:57:28.901747144   26718 handshaker.cc:212]          handshake_manager 0x35d2170: error="No Error" shutdown=0 index=2, args={endpoint=0x33b3220, args=0x33acc00 {size=11: grpc.primary_user_agent=grpc-node/1.16.1, grpc.client_channel_factory=0x7f406739f520, grpc.channel_credentials=0x33b2480, grpc.server_uri=dns:///firestore.googleapis.com:443, grpc.channelz_channel_node_creation_func=0x7f406712be50, grpc.http_connect_server=firestore.googleapis.com:443, grpc.default_authority=firestore.googleapis.com:443, grpc.http2_scheme=https, grpc.security_connector=0x34ca5e0, grpc.subchannel_address=ipv4:127.0.0.1:3128, grpc.auth_context=0x33c1b20}, read_buffer=0x338e500 (length=0), exit_early=0}
I1215 13:57:28.901762578   26718 handshaker.cc:245]          handshake_manager 0x35d2170: handshaking complete -- scheduling on_handshake_done with error="No Error"
I1215 13:57:28.901858190   26718 subchannel.cc:656]          New connected subchannel at 0x3529ab0 for subchannel 0x35c0250
test-topic

Squid log:

1544911049.455    635 172.17.0.1 TCP_TUNNEL/200 4455 CONNECT firestore.googleapis.com:443 - HIER_DIRECT/216.58.195.74 -

It seems I must set the HTTPS_PROXY variable for the credentials to go through the proxy, but it doesn't work with plain HTTP.

jkwlui · 2019-01-25T20:40:26Z

@hiranya911 We've release 0.21.0 which addressed the issue with running behind a proxy. Can you try now and see if it resolves it? Thanks!

jkwlui · 2019-01-25T21:00:59Z

@stephenplusplus I understand you have a test utility for proxy issues - can you please take a look at this? Thanks :)

JustinBeckwith · 2019-02-05T16:40:08Z

@stephenplusplus will help with verifying this

stephenplusplus · 2019-04-29T21:20:02Z

I believe @ajaaym found a solution:

can you please try setting both env variable https_proxy & HTTPS_PROXY GRPC uses https_proxy and authentication which goes through normal node request uses HTTPS_PROXY.

amammay · 2019-06-17T15:19:23Z

@JustinBeckwith @jkwlui with the release of 2.1.1 i can no longer hit firestore over a proxy. i have tried setting https_proxy, HTTPS_PROXY, http_proxy, HTTP_PROXY. It looks like 2.1.1 added "@grpc/grpc-js": "0.4.0". and from taking a quick look in that repo i dont see anything about grpc-js accepting proxy support? What do either of you think would be the best way to move forward?

stephenplusplus · 2019-06-17T16:26:16Z

That's unfortunate, I will look into that. I believe you could use a workaround where you provide the proxy endpoint programmatically in the constructor. @schmidt-sebastian is that possible? In other languages, it would look something like:

new Client({ apiEndpoint: 'https://localhost:8080' })
new Client({ host: 'localhost', port: '8080' })

schmidt-sebastian · 2019-06-17T17:07:31Z

The recommended syntax is:

new Client({ host: 'localhost:8080' })

apiEndpoint and servicePath should also work.

Note: We now also support non-SSL endpoints directly:

new Client({ host: 'localhost:8080'; ssl: false });

amammay · 2019-06-17T17:31:04Z

so that would work fine, but im behind a corporate proxy and need to pass in proxy authentication credentials. whenever i do so i get a Value for argument "settings.host" is not a valid host.

@schmidt-sebastian @stephenplusplus

in addition im getting this issue across other google apis projects like dialogflow or probably anything that is using grpc-js.

schmidt-sebastian · 2019-07-08T16:07:11Z

@amammay How do you pass these credentials? If they are HTTP headers, you can pass them in a such

const firestore = new Firestore({
  customHeaders: {
    "X-Name": "Value"
  }
});

amammay · 2019-07-08T22:51:41Z

@schmidt-sebastian well passing in the credentials would be of the format of

http://[user]:[pass]@hostname:port

and for the firebase settings you can only pass in host and port as https://github.com/googleapis/nodejs-firestore/blob/master/types/firestore.d.ts#L56

schmidt-sebastian · 2019-07-12T22:01:55Z

FWIW, we also support servicePath and apiEndpoint, which are passed directly to the underlying layer. I haven't yet been able to confirm that the support passing credentials as you suggested.

schmidt-sebastian · 2020-01-03T00:29:26Z

@hiranya911 Is this still an issue? We are now using Node's HTTP2 module for our networking.

hiranya911 · 2020-01-03T20:08:49Z

I've never tried this scenario with a proxy that requires authentication. Does the new implementation still support the https_proxy environment variable?

fatherOfLegends · 2020-01-11T01:57:30Z

I have not been able to successfully proxy any Firestore calls. I can proxy the token call but no requests to Firestore actually hit the proxy. I am using the latest version.

thebrianchen · 2020-01-16T00:38:04Z

The admin SDK uses grpc-js, which currently does not support proxies (issue). However, the native grpc library does support proxies using HTTP_PROXY.

One way to circumvent this limitation would be to provide a the native GRPC implementation to the admin SDK via the constructor argument as detailed in this comment, which also details why we use grpc-js instead of native grpc.

fatherOfLegends · 2020-01-16T01:26:14Z

@thebrianchen thanks for the tip. I gave this a shot, configured my app as the comment suggests, and using Charles proxy on my machine and I see the auth call hit the proxy but I do not see any calls to my Firestore database that hit the proxy.
"firebase-admin": "8.8.0"
"grpc": "1.24.2"

fatherOfLegends · 2020-01-16T01:34:05Z

@thebrianchen my apologies, I am hitting the proxy now I was using the wrong casing for the environment variable. HTTP_PROXY !== http_proxy

fatherOfLegends · 2020-01-16T01:34:28Z

@thebrianchen thank you!!

stephenplusplus · 2020-04-01T14:59:10Z

@zamnuts would you mind giving this a glance when you have a chance? From your recent experience with proxy support inside grpc-node, maybe we've resolved this already? And if not, do you know what issues are standing in the way?

schmidt-sebastian · 2020-04-01T16:19:20Z

@grpc/grpc-js 0.7.x adds supports for the HTTP_PROXY environment variable. We need to update our dependency and then we should be able to close this issue.

schmidt-sebastian · 2020-04-01T20:51:06Z

If you update your dependencies, you should now be using @grpc/[email protected].

This was referenced Dec 16, 2018

HTTPS_PROXY support does not work with an HTTP proxy googleapis/google-auth-library-nodejs#571

Closed

HTTPS Over HTTP Proxy Tunneling Issue googleapis/node-gtoken#131

Closed

JustinBeckwith added triage me I really want to be triaged. 🚨 This issue needs some love. labels Dec 16, 2018

yoshi-automation added the 🚨 This issue needs some love. label Jan 25, 2019

jkwlui added priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. and removed 🚨 This issue needs some love. triage me I really want to be triaged. labels Jan 25, 2019

yoshi-automation added triage me I really want to be triaged. 🚨 This issue needs some love. labels Jan 25, 2019

jkwlui mentioned this issue Jan 25, 2019

Release @google-cloud/firestore v0.21.0 #530

Merged

jkwlui removed 🚨 This issue needs some love. triage me I really want to be triaged. labels Jan 25, 2019

jkwlui self-assigned this Jan 25, 2019

yoshi-automation added the 🚨 This issue needs some love. label Jan 26, 2019

JustinBeckwith assigned stephenplusplus Feb 5, 2019

stephenplusplus closed this as completed Apr 29, 2019

JustinBeckwith reopened this Jun 17, 2019

JustinBeckwith removed 🚨 This issue needs some love. priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. labels Jun 17, 2019

JustinBeckwith added type: question Request for information or clarification. Not an issue. and removed type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. labels Jun 17, 2019

bcoe assigned alexander-fenster Jun 17, 2019

bcoe added the gapic label Jun 17, 2019

snyk-bot mentioned this issue Dec 5, 2019

[Snyk] Upgrade @google-cloud/firestore from 0.17.0 to 0.21.0 dyna-dot/g.gl_communityresources#8

Open

google-cloud-label-sync bot added the api: firestore Issues related to the googleapis/nodejs-firestore API. label Jan 30, 2020

schmidt-sebastian closed this as completed Apr 1, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Firestore Client Does Not Work Behind a Proxy #493

Firestore Client Does Not Work Behind a Proxy #493

hiranya911 commented Dec 15, 2018

hiranya911 commented Dec 15, 2018

jkwlui commented Jan 25, 2019

jkwlui commented Jan 25, 2019

JustinBeckwith commented Feb 5, 2019

stephenplusplus commented Apr 29, 2019

amammay commented Jun 17, 2019

stephenplusplus commented Jun 17, 2019

schmidt-sebastian commented Jun 17, 2019 •

edited

Loading

amammay commented Jun 17, 2019 •

edited

Loading

schmidt-sebastian commented Jul 8, 2019

amammay commented Jul 8, 2019

schmidt-sebastian commented Jul 12, 2019

schmidt-sebastian commented Jan 3, 2020

hiranya911 commented Jan 3, 2020

fatherOfLegends commented Jan 11, 2020 •

edited

Loading

thebrianchen commented Jan 16, 2020

fatherOfLegends commented Jan 16, 2020 •

edited

Loading

fatherOfLegends commented Jan 16, 2020

fatherOfLegends commented Jan 16, 2020

stephenplusplus commented Apr 1, 2020

schmidt-sebastian commented Apr 1, 2020

schmidt-sebastian commented Apr 1, 2020

Firestore Client Does Not Work Behind a Proxy #493

Firestore Client Does Not Work Behind a Proxy #493

Comments

hiranya911 commented Dec 15, 2018

Environment details

Steps to reproduce

GRPC debug log

HTTP proxy log (Squid proxy access log)

hiranya911 commented Dec 15, 2018

jkwlui commented Jan 25, 2019

jkwlui commented Jan 25, 2019

JustinBeckwith commented Feb 5, 2019

stephenplusplus commented Apr 29, 2019

amammay commented Jun 17, 2019

stephenplusplus commented Jun 17, 2019

schmidt-sebastian commented Jun 17, 2019 • edited Loading

amammay commented Jun 17, 2019 • edited Loading

schmidt-sebastian commented Jul 8, 2019

amammay commented Jul 8, 2019

schmidt-sebastian commented Jul 12, 2019

schmidt-sebastian commented Jan 3, 2020

hiranya911 commented Jan 3, 2020

fatherOfLegends commented Jan 11, 2020 • edited Loading

thebrianchen commented Jan 16, 2020

fatherOfLegends commented Jan 16, 2020 • edited Loading

fatherOfLegends commented Jan 16, 2020

fatherOfLegends commented Jan 16, 2020

stephenplusplus commented Apr 1, 2020

schmidt-sebastian commented Apr 1, 2020

schmidt-sebastian commented Apr 1, 2020

schmidt-sebastian commented Jun 17, 2019 •

edited

Loading

amammay commented Jun 17, 2019 •

edited

Loading

fatherOfLegends commented Jan 11, 2020 •

edited

Loading

fatherOfLegends commented Jan 16, 2020 •

edited

Loading