[Azure Container Apps - Azure Container Apps] API Review

[azure-sdk]

New API Review meeting has been requested.

Service Name: Azure Container Apps - Azure Container Apps
Review Created By: Rajneesh Mitharwal
Review Date: 09/05/2024 08:00 AM PT
Release Plan: 1418
PR: #29258
Hero Scenarios Link: here
Core Concepts Doc Link: here

Description: 1. Customer create a code interpreter or dedicated session pool via ARM APIs (control-plane resources)
2. Customer can expose their own APIs or other protocols via dedicated session pool. No 1P data-plane REST APIs for that.
3. We expose 1P data-plane REST APIs for code interpreter scenarios and providing support for two langs (Node JS & Python) out of box

Detailed meeting information and documents provided can be accessed here
For more information that will help prepare you for this review, the requirements, and office hours, visit the documentation here

[mikekistler]

Notes from API Review 9/5/24

PR is actually
#30424

• How are versions of the language runtimes managed?

  • Version is fixed based on when the session is created
  • To update the runtime version the customer must create a new session

• How did you decide to split between mgmt plane and data plane?

  • These operations need low execution time
  • File management could be handled with BYOS pattern
  • Data plane operations don't follow Azure data-plane guidelines
  • Where is subscription used? Should remove it if not used
  • Abstracting the subscription and resource group away and replace with opaque identifer. Then you can follow all the data-plane guidelines and use the TypeSpec dataplane libraries.

• DNS names -

• Post action should have :action as the last segment of the URL

  • And this must be idempotent -- how does that work?

• Get can download any file that was produced by the code execution

  • These are designed to be similar to the OpenAI files interfaces
  • This will be the main usage scenario
  • Okay. You can be a superset of this but not a subset
    • I.e. an app written to their API should be able to use your API more or less without change

• Need to design this for the future

Please address the above issues and then schedule a follow-up since this is for GA.

[mikekistler]

Pls provide a link to the OpenAI files APIs that this API should be compatible with.

[BlackRider97]

Pls provide a link to the OpenAI files APIs that this API should be compatible with.

https://platform.openai.com/docs/api-reference/files

[BlackRider97]

Thanks for capturing the feedback over GitHub issue and data-plane APIs review PR apart from discussion we had in the meeting. We tried to capture all responses here for all top concerns/questions were asked.

Setting up common ground on what we mean by a session under session pool

• A session is a sandboxed environment that runs your code or application. Each session is isolated from other sessions and from the host environment with a Hyper-V sandbox.
• A customer can run multiple code execution API calls with-in a session maintaining in-process code state of a session (example, define a method in first call and call the method in second call) which is backed Jupyter kernel behind the scenes.
• Additionally, customer can download the output generated code executions or upload/list etc. of files to do any file operation on them in their code execution. Example like upload an image to generate a thumbnail, execute code to do processing and download it.

1. Make sure there is a plan to support language runtime version

[Response]

Agreed and helpful feedback. We will have a clear documentation on which runtime customer will get and what's our plan for supporting them and updating it in future. In future, we will also run-time stack version under created session pool for in-place information for end-customer.

2. Why did we choose data-plane APIs for code execution and related operations instead of the management plane?

[Response]

We imagine session as a data-plane resource of session pool ARM resource which gives more flexibility to us like low-latency code execution in a session like current end-to-end execution for Bing Copilot session is happening within 80 msecs. The volume of code execution calls and number of sessions created are in millions in a day so creating a persisted ARM resource for a session is setup overhead for customers. Enabling WebSocket like other TCP protocols would be possible with data-plane implementation like we are working on GPU enabled Dynamic sessions (separate from this review). Since primary use case is around LLM agents so we need to provide a support for uploading large size files for eco-system parity while ARM limit that with ~10 MB size. Storing these files directly only Storage is not intended experience for customers as often it includes sensitive data and typically meant for one-time use.
We are using Azure data-plane RBAC (remote PDP call on data action validation) on data-plane APIs and don't have any different authentication and authorization mechanisms.

3. FQDN should follow ARM guidelines for data-plane endpoints

[Response]

Currently, the format is: https://eastasia.dynamicsessions.io/subscriptions/cabb33e6-3c92-4907-9d7c-80c7ca9ac327/resourceGroups/aca-session-prod-deployment-1/sessionPools/my-session-pool which we return as "data-plane endpoint" provided by the service in session pool ARM resource response, should be treated as a string, and customers should configure the sessions data-plane SDKs endpoint as it is as provided by us. We will make changes in future to have a unique FQDN for each pool to further eliminate the need of serviceRoot as path parameters as mentioned by Azure DNS team here.
Earlier in the PR, it was mentioned as API path params, but we have updated the PR to reflect the correct state. I am guessing it was creating confusion on control-plane looking endpoint for data-plane API calls.

4. Either follow Azure guidelines for data-plane resources or align with OpenAI APIs like we did for file operations APIs

[Response]

We internalized it and agreed to follow Azure guidelines for 1 code execution API and 5 file operation APIs (get metadata of a file, list of metadata of files, download, upload, delete a file) with keeping current Preview customers supported as it is. Code execution API can respond back with 200 OK if execution type is sync or 202 if execution type is async with additional location header information and GET API for long-poll etc. as per following LRO pattern.
Currently sessionId is query param instead of path param to have parity with our other capability 'user bringing up their own container image with their own REST APIs' so having query param gives us a deterministic way to extract Session ID.

5. Other feedback

Don't use foundational operations—refer to examples in the Playground or recently reviewed TypeSpec services.
[Response] Updated the PR now. If there is further scope of improvement then can do that.
The server endpoint is missing.
[Response] Added now. Customer should treat this as a string which is provided by us in ARM session pool response.

Avoid using Subscription ID as a model if it's already defined as a path parameter.
[Response] Updated in PR.

[azure-sdk]

New API Review meeting has been requested.

Service Name: Azure Container Apps
Review Created By: Rajneesh Mitharwal
Review Date: 10/02/2024 09:00 AM PT
Release Plan:
PR: #30424
Hero Scenarios Link: Not Provided
Core Concepts Doc Link: Not Provided

Description:

Detailed meeting information and documents provided can be accessed here
For more information that will help prepare you for this review, the requirements, and office hours, visit the documentation here