API key proxy

[ampsonic]

Will this package work with an api proxy to protect the api key, or is the warning more of an”don’t use this package with a shipping app” warning?

[cgaaf]

I imagine the optimal use of this library is for swift server development to create an api proxy.

Not sure how one would use it securely on the client side.

[davit-b]

Yea that's the whole thing I don't get.

There are two options if you use this library
A. Route all requests from client-app through a proxy server you build
B. Using Cloudkit to rotate out API keys to client-apps

Option A...I would build the server in Typescript since the TS community have much more support for OpenAI, tokenizer libs...etc

For Option B... I feel it should be discussed by the main contributors of the package. Why a security practice not documented anywhere? Do they just know that everyone will break the security practice anyway, and turn a blind eye to it?

[davit-b]

What I mean is, I've come to the same conclusion as you.

But I wish the owners of the repo just wrote at the top in big letters "This package is to build a Swift written server that does OpenAI API things"

[ppoh71]

oh man, that's exactly where I am now...

[leventmolla]

I am more inclined to use a solution that registers a separate OpenAI PI key for each user and possibly stores this securely on iCloud for each user. Thus the application would load the user's API key from iCloud (encrypted?) and use it for each session.
I am unaware of a few things, so what I suggest might be impossible: 1) I did not check OpenAI documentation to see if they allow creating new API keys with an API call and whether there is a Swift implementation for this anyone has done. 2) I think there is a way to store encrypted values on iCloud

[StefanBristol]

You can save keys like this encrypted in Apple keychain on the device. The user can enter their own key. 👍

[kalafus]

Please suggest language for an informative disclaimer and links to relevant documentation in a pull❣️

[ppoh71]

I wanted to start a new discussion, but the I found this one. Of course I am not alone....

I still deployed my app without a proxy and used the openai API directly in my app. After few days in the AppStore my API key got compromised and was used for by Chatgpt4 request. Luckily I had a set a limit and learned a lesson.

I wanted to start a discussion about what best solution for the recommended server-side communication between Openai exists, so your API key doesn't get exposed to the client.

Some solutions using Firebase / Google Cloud Functions are very safe but cannot stream back the results word by word, which is a UX problem for my app.

Setting up for own server with node.js or python could be a solution but adds another layer of complexity to the stack
. There are many solutions out there, but at the moment I am not sure which way to go when it comes to the macpaw/openai client and a secure proxy with stream capabilities.

I was even thinking of putting the macpaw client on a server-side swift app with vapor, but that is probably a wild idea.

[davit-b]

A server has two purposes for existence

1. Get securely requested to do stuff
2. Return a JSON to a client.

If that's the case, what's the advantage of writing and maintaining a server in Swift, when there are 1000x more documentation, tutorials, guides, medium posts, libraries for servers in other languages.

OpenAI has great support in Python and TS (langchain, token utility libs), and Client->Server interaction has great support in JS/TS (auth libs, middleware, tons of FaaS provides like Firebase and Amplify)

Read my architecture recommendation below.

[ppoh71]

Server-side Swift is course a stupid idea, the only reason I was thinking about it was to implement the MacPaw client on a server so I don't have to write my API on the server side again.
Meaning that when I write the Firebase Function to communicate with OpenAI and my iOS Client what do I need the MacPaw client for? I will probably be ending up using a URLSession communication with Firebase and that is it.
Anyhow thank you very much for your dedicated answer below, which helped me choose Firebase over all other solutions.
Streaming is the only thing I have to still figure out...

[davit-b]

Hey I actually just got a streaming POC working.
Firebase HTTP function, AppCheck protected, and Streaming GPT response back to iOS client.

Pretty simple, if you share a way to contact you I can show you how.

[ppoh71]

looks very promising, you can write me here: [email protected]
also working on a solution..

[vladimirDarksy]

@davit-b stream are still work? I will be very thankful if u send me the solution of this stuff 😇
[email protected]

[davit-b]

@ppoh71

Ok, here's what you and everyone else need to do.

You need a server. A Lambda, Google/Firebase Function, Vercel ServerActions (easy-mode function).
You need a client software for iOS that sends requests to this server. You can go barebones with REST and insecure
Client ---REST---> Server
or you can use the client sdks and do something more secure.
Client ---REST under the curtains ---> Server

Now, you still have security problem. Because your Server API is unsecured. How do you secure your client->server?
If you have your users create an account and login in to use the app:
Have an authenticated API endpoint that requires the user to be logged in to make a request to that endpoint (this is easy with Firebase or AWS Amplify, and or ClerkJS + Vercel Functions...but Clerk unfortunately have no iOS SDK)
If your want your users to use the app without logging in and making an acocunt:
This is harder. You have to ensure that every client->server call is ONLY coming from YOUR iOS app, and is ONLY happening by the App itself (not them using a client token and warping the request).
This is called App Attestation. Where an API call attests that it's indeed coming from an app. iOS supports an older way (deviceCheck) and a newer way (appcheck). You can hard-mode implement yourself or use higher-level libraries.

And even after attesting the app, a crafting pen tester can still subvert that, so you need to rate limit your API, and then also rate limit your server->OpenAI.

If you just want a solution that works, just do the following.

1. Setup Firebase functions,
   Use Function v2, callables (secured). Setup Firebase app check, and secure functions with app check verifying calls only.
   a. If you have login, great, make the function calls login authenticated.
   b. If you don't have login...app check is better than nothing.
2. Setup firebase function client on iOS, so your iOS app calls the function.
3. The function makes the API call to OpenAI with your API Key.
4. Set up rate limiting on the Function, because...you never know.
5. Set up account limit in OpenAI on the usage for your production api key.
6. Now your Client -> Server API -> OpenAI stuff.

So essentially you get iOS->Server API security, API rate limiting, OpenAI rate limiting.

You lose streaming, but...you can go through the pain in the ass of setting up your API to be streaming and then forward the stream to client. Figure out how to do with that a Function.

Also, you can swap our Firebase for Vercel, but then you lose the easy login and app check features( have to handroll), or you can go AWS Amplify (and you lose the app check and easy firestore/storage...etc). Or host your own Express server on Railway and then you go through a different nightmare. Pick your nightmare.

The reality is the amount of setup you have to do to make this secure means if you are handrolling this yourself (passing JWTS) and what not, god bless you, you don't need to read anything I am writing because you are a better dev than me.
Otherwise, you need to use a lot of libraries that take away a ton of the work of checking things, authorization, authenticating calls, device attestation...etc.

There is a LOT of work to be done to use OpenAI securely from an iOS app. I understand why the authors decided to leave the documentation section regarding security architecture empty. Unfortunately, I don't have the time to do a full PR and contribute, I only have the time to give you this little snippet to help you along the way.

[AlexWaldron1]

I keep getting internal failure error when I run my firebase function, @ppoh71 do you have any advice on how I can debug?

[ppoh71]

yes you can debug locally. See here: https://firebase.google.com/docs/functions/local-emulator
Run the emulator locally from the Firebase CLI with: firebase emulators:start and use the given local url as endpoint.
The emulator comes with a good debugger (http://127.0.0.1:4000/logs).

[drosenstark]

This is great and helpful but DOES IT USE THIS LIB AT ALL? Or is this lib just for prototyping and you just build REST calls?

@ppoh71

Ok, here's what you and everyone else need to do.

[ppoh71]

I don’t use macpaw/openai anymore. Since I run it on a server (firebase) I use the openai api directly (via node.js)

[davit-b]

@drosenstark Of course it doesn't use this lib. this lib is a perfect recipe for getting your API keys stolen. You need a proxy server. Either host your own, or google the SaaS providers today they do OpenAI proxying as a service.