Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Describe B&A payload size optimization options #1183
Describe B&A payload size optimization options #1183
Changes from all commits
443586417420cd5b49aaaa1c2c72d0b0f47File filter
Filter by extension
Conversations
Jump to
There are no files selected for viewing
Protected Audience Browser Bidding & Auction API
Background
This document seeks to propose an API for web pages to perform Protected Audience auctions using Bidding and Auction (B&A) servers running in Trusted Execution Environments (TEE), as was announced here. This document seeks to document the web-exposed JavaScript API. The browser is responsible for formatting the data sent to the B&A servers using the B&A server API documented in this explainer.
Steps to perform a Protected Audience auction using B&A
Step 1: Get auction blob from browser
To execute an on-server Protected Audience auction, sellers begin by calling
navigator.getInterestGroupAdAuctionData()with returns aPromise<AdAuctionData>:The
sellerfield will be checked to ensure it matches thesellerspecified in the auction configuration passed torunAdAuction()with the response. ThecoordinatorOriginselects which set of TEE keys should be used to encrypt this request. ThecoordinatorOriginmust be a coordinator that is known to Chrome. TherequestSizeandperBuyerConfigfields are described in more detail in the Request Size and Configuration section below.The returned
auctionBlobis a Promise that will resolve to anAdAuctionDataobject. This object containsrequestIdandrequestfields. TherequestIdcontains a UUID that needs to be presented torunAdAuction()along with the response. Therequestfield is aUint8Arraycontaining the information needed for the ProtectedAudienceInput in aSelectAdB&A call, encrypted using HPKE with an encryption header like that used in OHTTP. The encryption is done using public keys the browser fetches from a trusted endpoint that correspond with private keys only shared with B&A servers running in TEEs. Therequestis derived from the interest groups stored in the browser to be provided to the B&A API. The request includes the contents of the interest groups, and information that the B&A server can pass unmodified inbrowserSignalsfor bidding likejoinCount,bidCount, andprevWins. More details on the request format are given in the appendix.The
selleris required to have its site (scheme, eTLD+1) attested for Protected Audience API. Please see the Privacy Sandbox enrollment attestation model.Step 2: Send auction blob to servers
A seller’s JavaScript then sends auctionBlob to their server, perhaps by initiating a Fetch using a PUT or POST method with auctionBlob attached as the request body:
fetch('https://www.example-ssp.com/auction', { method: "PUT", body: auctionBlob, … })Their server then passes the blob off to B&A servers which conduct on-server auctions. Several on-server auctions may take place, e.g. one for each ad slot on the page. Each on-server auction returns an encrypted response blob in the format specified in the appendix.
Step 3: Get response blobs to browser
These response blobs are sent back to the browser. A seller’s JavaScript can Fetch them back to the browser, perhaps as the response body to the Fetch initiated in Step 2. This Fetch is initiated with a flag to prepare the browser to look for
Ad-Auction-ResultHTTP response headers:fetch('https://www.example-ssp.com/auction', { adAuctionHeaders: true, … })Note that
adAuctionHeadersonly works with HTTPS requests.Response blobs can also be retrieved using an
iframenavigation by specifying theadAuctionHeadersattribute on the iframe element. As with the Fetch flag, theadAuctionHeadersiframe attribute prepares the browser to look forAd-Auction-ResultHTTP response headers:For each response blob sent back to the browser, the seller’s server attaches a response header containing the base64url encoded (RFC 4648 section 5) SHA-256 hash of the response blob:
Multiple hashes can be included in a response by either repeating the header or by specifying multiple hashes separated by a
,character. Sois equivalent to
and both versions should be accepted.
It should be noted that the
fetch()request usingadAuctionHeaderscan also be used to sendauctionBlob(e.g. in the request body) and receive the response blob (e.g. in the response body).Step 4: Complete auction in browser
Now the auction can be completed by passing the response blob to
runAdAuction()as part of a specially configured auction configuration:The browser verifies it witnessed a Fetch request to the
seller’s origin with"adAuctionHeaders: true"that included anAd-Auction-Resultheader with hash ofresponse_blob. It also checks that theresponse_blobcorresponds to thenavigator.getInterestGroupAdAuctionData()request with the providedrequestIdthat was sent from the same frame. Since the Fetch request required HTTPS which authenticates the seller’s origin, this verification authenticates that the seller produced the response blob.runAdAuction()then proceeds as if the auction happened on device. This specially configured auction configuration can be used for single seller auctions or as a component auction configuration for multi-seller auctions (in this case thegetInterestGroupAdAuctionData()call must include atop_level_sellerfield that must later match the top-level seller passed torunAdAuction()). To facilitate parallelizing on-device and on-server auctions, theserverResponsecould be a Promise that resolves to aUint8Arraycontaining the blob later.Device Orchestrated Multi-Seller Auctions
Auctions run using the Bidding and Auction servers can be mixed with on-device auctions using component auctions. The top-level scoring script will decide between the top scoring bidders from the component auctions.
Note that since the
serverResponsefield in the config is a promise it is possible to run the on-device auction in parallel with the B&A auction.Request Size and Configuration
Request Size Controls
The
requestSizefield provided tonavigator.getInterestGroupAdAuctionData()can be used to specify the maximum size of the returned request. If theperBuyerConfigfield is not present and the blob fits into a size bucket smaller thanrequestSizethen that size will be used instead.If the
perBuyerConfigfield is specified and non-empty, the returned encrypted blob will be exactlyrequestSizebytes long unless there was an error. If an error occured then the returned blob will be 0 size and the function may throw an error.Buyer Controls
If the
perBuyerConfigfield is specified then the blob will only include the interest groups for the buyers specifically listed. IfrequestSizeis not specified then the sum of thetargetSizeof each buyer will be used for therequestSize. Interest groups for each buyer will be added to the request -- in decreasing order by the interest group'spriorityfield -- until the next interest group does not fit.Space can be allocated between different buyers in several different modes, depending on what options are specified in the
perBuyerConfig:Equally - Space is divided equally between buyers when
targetSizeis not specified.Proportionally - Space is assigned to buyers using
targetSizeas the weight. So a buyer with atargetSizeof 2 will can use twice as much space as a buyer with atargetSizeof 1. If all buyers have atargetSizespecified then proportional mode is used.Fixed/Mixed - Space is first allocated to buyers with
targetSizespecified. They get up totargetSizebytes. The remaining space is divided equally between the buyers that did not havetargetSizespecified.The browser will first process buyers with
targetSizespecified before processing buyers without that field specified. Within either of these groups buyers are processed in random order. Space that was allocated to a buyer but was not used is divided up among remaining buyers based on the active allocation mode. The browser may, as an optimization, calculate and use the maximum length used by a buyer to better inform its allocation strategy.Privacy Considerations
The blobs sent to and received from the B&A servers can contain data that could be used to re-identify the user across different web sites. To prevent this data from being used to join the user’s cross-site identities, the data is encrypted with public keys whose corresponding private keys are only shared with B&A server instances running in TEEs and running public open-source binaries known to prevent cross-site identity joins (e.g. by preventing logging or other activities which might permit such joins).
There are however side-channels that could leak a much smaller amount of cross-site identity, the first and most obvious one being the size of the encrypted blob. To minimize this leakage further we suggest above that the encrypted blob be padded. For example, we can use exponential bucketing for request blobs. Keeping the number of buckets small, for instance, 7 will lead to <3 bits of leaked entropy per call to
getInterestGroupAdAuctionData(). Some mitigation techniques for the “1-bit leak” may be applicable here.To prevent repeated calls to
getInterestGroupAdAuctionData()leaking additional cross-site identity, we’ve decided, at least in the near-term, to make repeated calls togetInterestGroupAdAuctionData()return a blob of the same size. The tradeoff being that the blob cannot be dependent on inputs togetInterestGroupAdAuctionData(). For example whereasrunAdAuction()normally takes ainterestGroupBuyerslist dictating which buyers to include in the auction,getInterestGroupAdAuctionData()’s returned blob cannot depend on such a list and so must include all buyers that have stored interest groups on the device. This may cause larger blobs, and in turn slower network requests sending and receiving those blobs. This could in turn make the API more susceptible to abuse, e.g. fake calls tojoinAdInterestGroup()bloating the blob size with spam interest groups.Another way to prevent the encrypted blob’s size from being a leak is to have the browser send the blob directly to the B&A servers instead of exposing it to JavaScript at all. Requiring this today has significant downsides:
This would hugely complicate the B&A server’s interactions and API, making adoption likely infeasible. The B&A API would no longer be a RESTful API as it would have to coordinate communication from both the browser and other servers (e.g. contextual auction server).
This would also require the on-device JavaScript to determine whether to send the Protected Audience request to the B&A server, perhaps at a time before it has the results of the contextual auction which might influence the decision. Without this information the device would have to send the encrypted blob for every ad request, even in cases where the contextual call indicated it was wasteful to do so.
Exposing size of the blob is a temporary leak that we hope to mitigate in the future:
As TEEs become more prevalent and adtechs gain experience operating and deploying them, we can reconsider whether the server that the browser sends the blob to could in fact be a trusted one operating in a TEE.
The size of the encrypted blob may shrink considerably, where padding may become a more effective privacy prevention (e.g. imagine fixed size blobs) and may introduce little overhead. We recently announced changes to the interest group update requirements that should facilitate significantly fewer interest groups. Having many fewer interest groups can greatly reduce the size of the encrypted blob. These same changes allow for more information to be stored in the real-time trusted bidding signals server and simply indexed with a small identifier kept in the interest group, again shrinking the interest groups.
Consented Debugging
The Bidding and Auction API offers a special "Consented Debugging" Mode where logs related an individual request are made available on the server. A developer can opt-in to this mode in Chrome by going to
chrome://flagsand enabling the "Protected Audience Consented Debug Token" feature with the same value that is set in the Bidding and Auction Server'sconsented_debug_tokenconfiguration. Chrome will send this value to the server which will produce additional logging information related to the request when it finds the value in the request matches the server value. Note that you should use a unique value for theconsented_debug_tokento avoid subjecting other servers involved in handling the request to additional load from logging.Testing with alternate Coordinators
The coordinators used by Chrome are controlled by two feature flags. The 'FledgeBiddingAndAuctionKeyURL' feature parameter controls the key URL used for the default coordinator (with the origin 'https://publickeyservice.gcp.privacysandboxservices.com'). The 'FledgeBiddingAndAuctionKeyConfig' feature parameter allows full control of the set of coordinator origins and their corresponding key URLs used by Chrome by specifying a JSON dict where the keys are the coordinator origins and the values are the corresponding key URL. You should only need to use one of these feature parameters, but if both are specified the 'FledgeBiddingAndAuctionKeyURL' will override the key URL for the default coordinator.
Examples of the command line for setting the coordinator for testing is shown below:
Set the default key URL to 'http://127.0.0.1:50072/static/test_keys.json':
Add a new coordinator for origin 'https://new.coordinator.example.com' with key URL of 'http://127.0.0.1:50072/static/test_keys.json'. Note that you still need to specify the default values:
Appendix
Request Blob Format
The schema below defines the format of the blob mentioned earlier. The blob is padded to either one of the 7 allowed sizes to reduce the cross-site identity leaked by its size. In the event the
requestcannot be created the resulting array will have zero length.Prior to encryption the
requestis encoded as CBOR with the following schema (specified using JSON Schema):{ "$schema": "https://json-schema.org/draft/2020-12/schema", "$id": "/BuyerInput/AuctionBlob", "type": "object", "properties": { "version": { "type": "number", "description": "Schema version used by this CBOR message. The schema version represented here is version 0." }, "compression": { "enum": ["gzip", "brotli", "none"], "default": "gzip", "description": "Compression format to use when compressing the elements of the interestGroups field" }, "publisher": { "type": "string" }, "generationId": { "type": "string", "format": "uuid", "description": "Random identifier produced by the browser that the server may use for rate limiting" }, "enableDebugReporting": { "type": "boolean", "default": "false", "description": "Controls whether the server should allow the forDebuggingOnly.reportAdAuctionLoss() and forDebuggingOnly.reportAdAuctionWin() APIs" }, "interestGroups": { "patternProperties": { "^https://": { "type": "string", "description": "CBOR encoded list of interest groups compressed using the method described in `compression`." } }, "additionalProperties": false } }, "required": [ "version", "interestGroups", "publisher", "generationId" ] }where each compressed interest group item in
interestGroupsis an object where the key is the owner and value is a CBOR encoded list of the interest groups for that owner, compressed with the algorithm specified incompression.The schema for the CBOR encoding of the interest group (specified using JSON Schema) is:
{ "$schema": "https://json-schema.org/draft/2020-12/schema", "$id": "/BuyerInput/AuctionBlob/InterestGroup", "$defs": { "adRenderId": { "type": "string", "description": "A short identifier (up to 12 characters long) that can be used to reconstruct the ad object based on server-side information. Sent instead of the ad object." } }, "type": "object", "required": [ "name" ], "properties": { "name": {"type": "string"}, "biddingSignalsKeys": { "type": "array", "items": {"type": "string", "uniqueItems": true} }, "userBiddingSignals": {"type": "string"}, "ads": { "type": "array", "items": { "$ref": "#/$defs/adRenderId" } }, "components": { "type": "array", "items": { "$ref": "#/$defs/adRenderId" } }, "browserSignals": { "type": "object", "properties": { "joinCount": { "type": "number", "description": "Number of times the group was joined in the last 30 days." }, "bidCount": { "type": "number", "description": "Number of times the group bid in an auction in the last 30 days." }, "recency": { "type": "number", "description": "The most recent join time for this group expressed in seconds before the containing auctionBlob was requested." }, "prevWins": { "type": "array", "items": { "type": "array", "description": "Tuple of time-ad pairs for a previous win for this interest group that occurred in the last 30 days. The time is specified in seconds before the containing auctionBlob was requested.", "items": false, "prefixItems": [ { "type": "number" }, { "$ref": "#/$defs/adRenderId"} ] } } } } } }This roughly matches the specification of the interest group in the B&A API.
The request is framed with a 5 byte header, where the first byte is the value 0x02 and the following 4 bytes are the length of the request message in network byte order. Then the request is zero padded to a set of pre-configured lengths (TBD).
Example
The JSON equivalent of an example
auctionBlobwould look like this:{ "version": 0, "publisher": "https://foo.com", "generationId": "11111111-2222-3333-4444-555555555555", "interestGroups": { "https://owner1.com": "<bytes>", "https://owner2.com": "<bytes>" } }Where
<bytes>are filled in with gzip compressed CBOR encoded list of interest groups for each owner.The JSON equivalent of the interest group would look like the following example:
{ "name": "cars", "biddingSignalsKeys": ["key1", "key2"], "userBiddingSignals": "signals", "ads": [ "adRenderId", "adRenderId2" ], "components": [], "browserSignals": { "joinCount": 2, "bidCount": 0, "recency": 500, "prevWins": [ [2, "adRenderId"], [3, "adRenderId"] ] } }Response Blob Format
The response blob from a B&A auction contains an HPKE encrypted message containing the information from AuctionResult. This response has an encryption header like that used in OHTTP and serves as the response for the encryption context started by the
auctionBlobfromnavigator.getInterestGroupAdAuctionData. The response contains a framing header like the request and contains a blob of compressed data, using the same schema version and same compression algorithm as specified in theauctionBlob. The response needs to be padded to a set of sizes to limit the amount of information leaking from the auction.Prior to compression and encryption, the AuctionResult is encoded as CBOR with the following schema (specified using JSON Schema):
{ "$schema": "https://json-schema.org/draft/2020-12/schema", "$id": "/SellerFrontEnd/AuctionResponse", "type": "object", "$defs": { "errorDef": { "type": "object", "properties": { "code": { "description": "HTTP status code for the request.", "type": "number" }, "message": { "description": "Human readable error message.", "type": "string" } }, "additionalProperties": false }, "reportingUrlsDef": { "type": "object", "properties": { "reportingUrl": { "type": "string", "format": "uri", "description": "Reporting URL passed from sendReportTo in the corresponding reporting worklet." }, "interactionReportingUrls": { "description": "Map from events to beacon URLs for Fenced Frame Ads Reporting created by the corresponding reporting worklet. See https://github.com/WICG/turtledove/blob/main/Fenced_Frames_Ads_Reporting.md", "additionalProperties": { "type": "string", "format": "uri" } } } }, "winReportingUrlsDef": { "type": "object", "buyerReportingUrls": { "$ref": "#/$defs/reportingUrlsDef" }, "componentSellerReportingUrls": { "$ref": "#/$defs/reportingUrlsDef" }, "topLevelSellerReportingUrls": { "$ref": "#/$defs/reportingUrlsDef" } } }, "properties": { "adRenderURL": { "type": "string", "format": "uri", "description": "The ad that will be rendered on the end user's device." }, "components": { "type": "array", "description": "List of render URLs for component ads to be displayed as part of this ad", "items": { "format": "uri", "type": "string" } }, "interestGroupName": { "type": "string", "description": "Name of the InterestGroup (Custom Audience), the remarketing ad belongs to." }, "interestGroupOwner": { "type": "string", "format": "uri", "description": "Origin of the Buyer who owns the interest group that includes the ad." }, "biddingGroups": { "type": "object", "description": "Map from interest group owner to a list of interest groups that bid in this auction", "patternProperties": { "^https://": { "type": "array", "items": { "type": "number", "description": "Indices of interest groups in the original request for this owner that submitted a bid for this auction. This is used to update the bidCount for these interest groups. Note that we send the index instead of the name because that allows us to avoid compressing separately." } } }, "additionalProperties": false }, "score": { "type": "number", "description": "Score of the ad determined during the auction. Any value that is zero or negative indicates that the ad cannot win the auction. The winner of the auction would be the ad that was given the highest score. The output from ScoreAd() script is desirability that implies score for an ad." }, "bid": { "type": "number", "description": "Bid price corresponding to an ad." }, "isChaff": { "type": "boolean", "description": "Boolean to indicate that there is no remarketing winner from the auction. AuctionResult may be ignored by the client (after decryption) if this is set to true." }, "winReportingUrls": { "$ref": "#/$defs/winReportingUrlsDef" }, "error": { "$ref": "#/$defs/errorDef" } } }There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As a seller concerned about request size,
perBuyerConfigreturning the exactrequestSizeis quite suboptimal. While it's highly desirable to fairly fit multiple buyers on the request, we would prefer to take the min(actual_size, request_size) instead.Could we satisfy the privacy constraints in a way that better optimizes for streamlined request payload size?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree that returning the exact request size is unfortunate, but returning even a bucketed size leaks significantly more data than we are comfortable with.
One alternative that we considered was allowing the config to be specified out of band per seller, and only allowed to change relatively slowly (preventing the leak from calling the API multiple times with the slightly different parameters). Unfortunately this can be worked around using multiple sellers that have configs that are slightly different. The sellers don't even have to be cooperating because the attack can be completely client-side.