Text Streaming not working when using Nginx-Ingress on Kubernetes

[smallbirds4]

Problem Description:
I have used the Nginx-Ingress service on my Azure Kubernetes Cluster, using the command:

helm upgrade --install ingress-nginx ingress-nginx --repo https://kubernetes.github.io/ingress-nginx --namespace ingress-nginx --create-namespace --set controller.service.loadBalancerIP=<ip-adress> --set controller.service.annotations."service\.beta\.kubernetes\.io/azure-load-balancer-health-probe-request-path"=/healthz

My kubernetes cluster is basically a frontend docker container interacting with a third-party service, streaming some text back to the frontend until a longer message has been completed. When I test the docker container locally it works perfectly, all the text gets streamed in real time word-by-word until the text is completed. When I put it on my kubernetes cluster behind the Nginx-Ingress, it only get the whole text chunk when all the has been completed (it works on my cellphone tho).

Investigation:
I tried to investigate the problem online, and looks like the issue could be related to the proxy-buffering. I found several suggestions online, about changing e.g.:

proxy-http-version 1.1
chunked-transfer-encoding on
proxy_pass http://localhost:3000/
proxy_request_buffering off

I have tried to implement them by using: kubectl apply -f setting.yaml - with this information in the .yaml file

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nnais-streamlit-ingress
  annotations:
    nginx.ingress.kubernetes.io/proxy-buffering: "off"
    nginx.ingress.kubernetes.io/proxy-http-version: "1.1"
    nginx.ingress.kubernetes.io/chunked-transfer-encoding: "on"
    nginx.ingress.kubernetes.io/proxy_pass: "http://localhost:3000/"
    nginx.ingress.kubernetes.io/proxy_request_buffering: "of"

It did not work, however. I have spend hours looking online, and have not found a simple explanation how to get this to work on my Azure Kubernetes cluster. Do I have to change a configuration before doing my helm installation, and how to instruct nginx-ingress to use that configuration using the helm installation. I have not been able to found an answers on how to do that.

[k8s-ci-robot]

This issue is currently awaiting triage.

If Ingress contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[longwuyuan]

/remove-kind bug

There is not enough info to understand the problem. The new issue template asks questions that help but yo have not answered any of those. You can make progress by answering questuons asked in the new issue template.

[smallbirds4]

Where can i find the new issue template?

[longwuyuan]

@smallbirds4 when you start to create a new issue, default template is rendered

[pbn4]

@smallbirds4 Have you found a solution to this issue? I have it too

[paolosalvatori]

Hi @smallbirds4 in the documentation for NGINX ingress controller annotations there is no nginx.ingress.kubernetes.io/chunked-transfer-encoding annotation. Said that, did you solve the issue?

[viktorshevchenko210]

@paolosalvatori It is on by default https://nginx.org/en/docs/http/ngx_http_core_module.html#chunked_transfer_encoding.
In config you can set this way

    annotations:
      nginx.ingress.kubernetes.io/configuration-snippet: |
        chunked_transfer_encoding "on";

[viktorshevchenko210]

@smallbirds4 Have you found a solution to this issue? I have tried literally everything but it doesn't work for me as well.

[viktorshevchenko210]

@smallbirds4 @paolosalvatori We have solved the issue.
The issue was that HTTP2 was enabled on AWS ALB. HTTP2 doesn't support chunked transfer encoding.
Also, NGINX doesn't support HTTP2. The only allowed values for proxy_http_version are 1.0 or 1.1.

https://greenbytes.de/tech/webdav/rfc7540.html#rfc.section.8.1.p.4

HTTP/2 uses DATA frames to carry message payloads. 
The chunked transfer encoding defined in Section 4.1 of [RFC7230] MUST NOT be used in HTTP/2.

[smallbirds4]

Thanks all of you. Unfortunately, none of the solutions are working :/ I am not using HTTP2

[Loki-Afro]

@smallbirds4
i guess there are 2 issues here
nginx.ingress.kubernetes.io/proxy_request_buffering: "of"
isn't it supposed to be
nginx.ingress.kubernetes.io/proxy-request-buffering: "off"?

also tehre are 2 nginx settings the newer https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_request_buffering
and https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffering

the later you already configured

[thien-nguyenthanh]

I resolved the same issue with Nginx by disable setting proxy-buffering: off. Not proxy-request-buffering

[rikatz]

/close

Apparently there is a fix here, thanks for the feedback @thien-nguyenthanh

[k8s-ci-robot]

@rikatz: Closing this issue.

In response to this:

/close

Apparently there is a fix here, thanks for the feedback @thien-nguyenthanh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[airedwin]

Thanks all of you. Unfortunately, none of the solutions are working :/ I am not using HTTP2

any update on this? i'm having a similar issue

[longwuyuan]

The recent release of controller uses NGINX v1.25 which supports HTTP2. Also chunking is on by default. So please check your use accordingly.