Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

--allow-unknown-fields has no effect; unknown fields are rejected #7936

Closed
howardjohn opened this issue Aug 15, 2019 · 10 comments
Closed

--allow-unknown-fields has no effect; unknown fields are rejected #7936

howardjohn opened this issue Aug 15, 2019 · 10 comments
Labels
question Questions that are neither investigations, bugs, nor enhancements

Comments

@howardjohn
Copy link
Contributor

Bug Template

Title: --allow-unknown-fields has no effect; unknown fields are rejected

Description:
When setting --allow-unknown-fields, unknown fields still seem to be rejected

When NOT setting --allow-unknown-fields, unknown fields seem to be accepted

Repro steps:
Download bootstrap.yaml and lds.yaml:
repro.zip

Run docker run -v ${PWD}:/tmp/envoy envoyproxy/envoy-dev:882a30677619856446f7e1b9d28c6ab319b21d1b -c /tmp/envoy/bootstrap.yaml

Output:


[2019-08-15 17:06:28.348][1][info][config] [source/server/configuration_impl.cc:62] loading 0 static secret(s)
[2019-08-15 17:06:28.348][1][info][config] [source/server/configuration_impl.cc:68] loading 0 cluster(s)
[2019-08-15 17:06:28.349][1][info][upstream] [source/common/upstream/cluster_manager_impl.cc:147] cm init: all clusters initialized
[2019-08-15 17:06:28.349][1][info][config] [source/server/configuration_impl.cc:72] loading 0 listener(s)
[2019-08-15 17:06:28.349][1][info][config] [source/server/configuration_impl.cc:97] loading tracing configuration
[2019-08-15 17:06:28.349][1][info][config] [source/server/configuration_impl.cc:117] loading stats sink configuration
[2019-08-15 17:06:28.349][1][info][main] [source/server/server.cc:506] all clusters initialized. initializing init manager
[2019-08-15 17:06:28.350][1][info][upstream] [source/server/lds_api.cc:60] lds: add/update listener 'virtualOutbound'
[2019-08-15 17:06:28.350][1][info][config] [source/server/listener_manager_impl.cc:775] all dependencies initialized. starting workers
[2019-08-15 17:06:28.351][1][info][main] [source/server/server.cc:523] starting main dispatch loop

Run docker run -v ${PWD}:/tmp/envoy envoyproxy/envoy-dev:882a30677619856446f7e1b9d28c6ab319b21d1b -c /tmp/envoy/bootstrap.yaml --allow-unknown-fields

Output:

[2019-08-15 17:06:55.556][1][info][config] [source/server/configuration_impl.cc:62] loading 0 static secret(s)
[2019-08-15 17:06:55.556][1][info][config] [source/server/configuration_impl.cc:68] loading 0 cluster(s)
[2019-08-15 17:06:55.556][1][info][upstream] [source/common/upstream/cluster_manager_impl.cc:147] cm init: all clusters initialized
[2019-08-15 17:06:55.556][1][info][config] [source/server/configuration_impl.cc:72] loading 0 listener(s)
[2019-08-15 17:06:55.556][1][info][config] [source/server/configuration_impl.cc:97] loading tracing configuration
[2019-08-15 17:06:55.556][1][info][config] [source/server/configuration_impl.cc:117] loading stats sink configuration
[2019-08-15 17:06:55.556][1][info][main] [source/server/server.cc:506] all clusters initialized. initializing init manager
[2019-08-15 17:06:55.557][1][warning][config] [source/common/config/filesystem_subscription_impl.cc:54] Filesystem config update failure: Protobuf message (type envoy.api.v2.DiscoveryResponse reason INVALID_ARGUMENT:fake_field: Cannot find field.) has unknown fields
[2019-08-15 17:06:55.557][1][info][config] [source/server/listener_manager_impl.cc:775] all dependencies initialized. starting workers
[2019-08-15 17:06:55.558][1][info][main] [source/server/server.cc:523] starting main dispatch loop
@mattklein123 mattklein123 added the question Questions that are neither investigations, bugs, nor enhancements label Aug 15, 2019
@mattklein123
Copy link
Member

I think this is fixed by #7857

@mattklein123
Copy link
Member

cc @htuch

@howardjohn
Copy link
Contributor Author

Great, thanks!

Just a note -- it seems that while with the latest the flag is inverted, with an older version envoyproxy/envoy-dev:829b905ca0fdc85233c3969247e53a62a52ac627, it is rejected with or without the flag

@htuch
Copy link
Member

htuch commented Aug 15, 2019

@howardjohn if it's working now, but inverted, I think this should be solved by #7857. Hope to merge this in the next day or so.

@howardjohn
Copy link
Contributor Author

@htuch it seems that with #7857 it is no longer inverted, but rather all flags allow-unknown-fields, allow-unknown-static-fields, and reject-unknown-dynamic-fields, and no flag, will now result in the config being rejected.

This should be reproducible with docker run -v ${PWD}:/tmp/envoy envoyproxy/envoy-dev:e1cd4cca46179562cf80de7f81774f4dc6354085 -c /tmp/envoy/bootstrap.yaml, where $PWD is the repro.zip from above.

Any ideas what is happening here?

@mandarjog
Copy link
Contributor

In 1.3 we are using allow-unknown-fields in istio 1.3 is that not working ?

@howardjohn
Copy link
Contributor Author

@mandarjog it appears to have been broken for a while

@htuch
Copy link
Member

htuch commented Aug 27, 2019

@howardjohn I'll look into this later today.

@htuch
Copy link
Member

htuch commented Aug 27, 2019

@howardjohn this works for me, see:

<master> htuch@htuch:~/src/envoy ⊧ bazel-bin/source/exe/envoy-static -c ./bootstrap.yaml
[2019-08-27 17:29:53.733][118238][info][main] [source/server/server.cc:242] initializing epoch 0 (hot restart version=11.104)
[2019-08-27 17:29:53.734][118238][info][main] [source/server/server.cc:244] statically linked extensions:
[2019-08-27 17:29:53.734][118238][info][main] [source/server/server.cc:246]   access_loggers: envoy.file_access_log,envoy.http_grpc_access_log,envoy.tcp_grpc_access_log                                                                                                                                                       
[2019-08-27 17:29:53.734][118238][info][main] [source/server/server.cc:249]   filters.http: envoy.buffer,envoy.cors,envoy.csrf,envoy.ext_authz,envoy.fault,envoy.filters.http.dynamic_forward_proxy,envoy.filters.http.grpc_http1_reverse_bridge,envoy.filters.http.header_to_metadata,envoy.filters.http.jwt_authn,envoy.filters.http.original_src,envoy.filters.http.rbac,envoy.filters.http.tap,envoy.grpc_http1_bridge,envoy.grpc_json_transcoder,envoy.grpc_web,envoy.gzip,envoy.health_check,envoy.http_dynamo_filter,envoy.ip_tagging,envoy.lua,envoy.rate_limit,envoy.router,envoy.squash                                                              
[2019-08-27 17:29:53.734][118238][info][main] [source/server/server.cc:252]   filters.listener: envoy.listener.http_inspector,envoy.listener.original_dst,envoy.listener.original_src,envoy.listener.proxy_protocol,envoy.listener.tls_inspector                                                                               
[2019-08-27 17:29:53.734][118238][info][main] [source/server/server.cc:255]   filters.network: envoy.client_ssl_auth,envoy.echo,envoy.ext_authz,envoy.filters.network.dubbo_proxy,envoy.filters.network.mysql_proxy,envoy.filters.network.rbac,envoy.filters.network.sni_cluster,envoy.filters.network.thrift_proxy,envoy.filters.network.zookeeper_proxy,envoy.http_connection_manager,envoy.mongo_proxy,envoy.ratelimit,envoy.redis_proxy,envoy.tcp_proxy
[2019-08-27 17:29:53.734][118238][info][main] [source/server/server.cc:257]   stat_sinks: envoy.dog_statsd,envoy.metrics_service,envoy.stat_sinks.hystrix,envoy.statsd                                                                                                                                                         
[2019-08-27 17:29:53.734][118238][info][main] [source/server/server.cc:259]   tracers: envoy.dynamic.ot,envoy.lightstep,envoy.tracers.datadog,envoy.tracers.opencensus,envoy.zipkin                                                                                                                                            
[2019-08-27 17:29:53.734][118238][info][main] [source/server/server.cc:262]   transport_sockets.downstream: envoy.transport_sockets.alts,envoy.transport_sockets.tap,raw_buffer,tls                                                                                                                                            
[2019-08-27 17:29:53.734][118238][info][main] [source/server/server.cc:265]   transport_sockets.upstream: envoy.transport_sockets.alts,envoy.transport_sockets.tap,raw_buffer,tls                                                                                                                                              
[2019-08-27 17:29:53.734][118238][info][main] [source/server/server.cc:271] buffer implementation: new
[2019-08-27 17:29:53.745][118238][info][main] [source/server/server.cc:331] admin address: 0.0.0.0:15000
[2019-08-27 17:29:53.828][118238][info][main] [source/server/server.cc:445] runtime: layers:
  - name: base
    static_layer:
      {}
  - name: admin
    admin_layer:
      {}
[2019-08-27 17:29:53.829][118238][info][config] [source/server/configuration_impl.cc:62] loading 0 static secret(s)
[2019-08-27 17:29:53.829][118238][info][config] [source/server/configuration_impl.cc:68] loading 0 cluster(s)
[2019-08-27 17:29:53.830][118238][info][upstream] [source/common/upstream/cluster_manager_impl.cc:161] cm init: all clusters initialized
[2019-08-27 17:29:53.830][118238][info][config] [source/server/configuration_impl.cc:72] loading 0 listener(s)
[2019-08-27 17:29:53.830][118238][info][config] [source/server/configuration_impl.cc:97] loading tracing configuration
[2019-08-27 17:29:53.830][118238][info][config] [source/server/configuration_impl.cc:117] loading stats sink configuration
[2019-08-27 17:29:53.831][118238][info][main] [source/server/server.cc:513] all clusters initialized. initializing init manager
[2019-08-27 17:29:53.838][118238][warning][config] [source/common/protobuf/message_validator_impl.cc:28] Unknown field: type envoy.api.v2.DiscoveryResponse reason INVALID_ARGUMENT:fake_field: Cannot find field.                                                                                                             
[2019-08-27 17:29:53.840][118238][info][upstream] [source/server/lds_api.cc:59] lds: add/update listener 'virtualOutbound'
[2019-08-27 17:29:53.840][118238][info][config] [source/server/listener_manager_impl.cc:789] all dependencies initialized. starting workers
[2019-08-27 17:29:53.859][118238][info][main] [source/server/server.cc:530] starting main dispatch loop
[2019-08-27T21:30:16.264Z] "GET / HTTP/1.1" 200 - 0 5072 10 - "100.118.30.65" "curl/7.64.0" "-" "localhost:15000" "-"
[2019-08-27T21:30:21.161Z] "GET /listeners HTTP/1.1" 200 - 0 31 0 - "100.118.30.65" "curl/7.64.0" "-" "localhost:15000" "-"

and

<master> htuch@htuch:~/src/envoy ⊧ curl localhost:15000/listeners
virtualOutbound::0.0.0.0:15001

It's detecting the unknown field and warning (as expected for the dynamic config), but it accepts it still.

@howardjohn
Copy link
Contributor Author

@htuch thanks! I didn't realize it was just warning. Looks good now, I think this can be closed. Thanks for the fix

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Questions that are neither investigations, bugs, nor enhancements
Projects
None yet
Development

No branches or pull requests

4 participants