Site.content.data for a specific [auction]

[patmmccann]

Type of issue

Right now https://docs.prebid.org/features/firstPartyData.html describes how to set ad unit specific first party data, but not how to set seller defined contextual segments along the lines of #6057 that would normally go in site.content.data

Ad-unit specific first party data might be valid if there is a video unit on the page and the contextual segments are about the video, not the text. It also might be relevant on various single page experiences with multiple types on content that fall into different groupings.

Possible solution

One solution is we set it on imp.ext.data.content . The issue here is adapters would all have to start looking for this data in a second place.

eg

imp": [{
		"id": "Video_Individual_Autoplay_SOff",
		"ext": {
			"data": {
				"adserver": {
					"name": "gam",
					"adslot": "/18190176,83852415/AdThrive_Video_Individual_Autoplay_SOff/53237591208f222c05b9a652"
				},
				"jwTargeting": {
					"playerID": "9F5pbYIL",
					"mediaID": "9F5pbYIL"
				},
				"content": [{
					"name": "www.iris.tv",
					"segment": [{
						"id": "ic_4146983"
					}, {
						"id": "ic_1670398"
					}, {
						"id": "ic_4260700"
					}, {
						"id": "ic_9534888"
					}, {
						"id": "ic_9954675"
					}, {
						"id": "ic_3393155"
					}],
					"ext": {
						"segtax": 501
					}
				}],

Alternative solution

It seems publishers might already have a workaround, setting the global fpd with these extra segments and then quickly unset it so other units aren't affected. Should we develop a utility function to more easily support this concept of temporary global fpd?

[gglas]

Discussed today, @bretg will look into and discuss the best way to handle this. Could be that we feature this in a second place.

[bretg]

Two use cases have been offered:

• video might have different site.content.data
• a page might have a split personality where some adunits have different site.content.data

Would it be reasonable to assume that these scenarios could be run as separate auctions? If so, we could add the ability to set ortb2 on requestBids:

pbjs.requestBids({
    bidsBackHandler: sendAdserverRequest,
    ...
    ortb2: {
       // for this auction only, deep-merge this block into any existing ortb2 settings sent on the adunit
       // or obtained through getConfig
    }
});

I think this would involve a change to the getConfig routine so when bid adapters call getConfig("ortb2"), they get a fully merged object.

[patmmccann]

Yes, This appears to be an excellent solution!

[dgirardi]

@bretg @patmmccann there are some technical problems with implementing an auction-aware config.getConfig('ortb2') - the same problems that afflict setBidderConfig: it can only be made to work when getConfig is called directly and from very specific places (currently adapter's buildRequests and interpretResponse).

For example, the fpd module currently calls getConfig('ortb2') in a way that cannot easily be made auction-aware. But more in general, it's a kind of "dark magic" that will cause us trouble in the future because it's very easy for a contributor (and a reviewer) to not fully understand/remember how it actually works. Another word for it is technical debt.

Here's my counter-proposal:

1. Implement @bretg's proposal with the caveats above (it will only work for direct calls from buildRequests and interpretResponse; it will silently and surprisingly fail elsewhere; it will be invisible to the fpd module);
2. Also make a fully merged ortb2 object available in the bidRequest (which would include global configuration; auction configuration, and the adUnit's ortb2Imp); then gradually ask adapters to use only that as the access point to FPD, and re-think how the fpd module should do enrichment / validation (it should probably run on each request, and not ask the publisher to manually refresh FPD data). Have a plan to eventually phase out bidRequest.ortb2Imp.

[bretg]

Thanks for pointing out the problems with the original proposal, @dgirardi.

Getting 70+ adapters to modify FPD again sounds like a pain. We spent a lot of time last year dealing with fallout from the last FPD implementation change.

Stepping back, Patrick's use case is mediatype-related. i.e. different site.content for video. So a more brute-force approach would be to define a parallel convention. e.g. setConfig({ortb2-video}). Adapters would need to read ortb2-video when they're building video requests.

So this needs to be discussed in committee -- if we have stomach for another wrenching change for 70 adapters, cool.

[dgirardi]

The change from config.getConfig('ortb2') to bidRequest.ortb2 could be done with little more than a search-and-replace, so not necessarily that painful for adapter maintainers (we could do it for them). However, removing the split we currently have between ortb2 and ortb2Imp would be more complicated.

[dgirardi]

Updated proposal:

• extend the pbjs.requestBids API to take in site-wide FPD as above:

  pbjs.requestBids({
   // ...
   ortb2: {
   // ....
   }
  })
  

• introduce a new layer between requestBids and the start of the auction, where the ortb2 parameter is transformed to be scoped by bidder:

  requestAuction({
      // ...
      ortb2: {
        standard: { /* ... * / },
        bidderA: { /* ... */ },
        /...
      }
  })
  

• Change the guidance for RTD submodules and refactor them to populate those fields instead of calling setConfig or setBidderConfig. They are working directly with the object passed to requestBids, so this new approach would simplify this part of the stack (this is the motivation for scoping FPD by bidders). The current system seems very fragile to me as I don't think it's guaranteed to work if there are multiple auctions running simultaneously.
• Just before calling into adapters' buildRequests(validBidRequests, bidderRequest), core will merge (in increasing order of priority) config.getConfig('ortb2') with ortb2.standard and ortb2.[bidder] as passed to requestBids and processed by RTD modules, and make the result available as bidderRequest.ortb2.
• Refactor all bid adapters that call config.getConfig('ortb2') to use bidderRequest.ortb2 instead, and update adapter guidance accordingly.
• For the special case of the PBS adapter, FPD merging will be done for each bidder. This boils down to adding the contents of requestBids({..}) here:

  Prebid.js/modules/prebidServerBidAdapter/index.js

  Lines 369 to 387 in 688d0b9

  	function addBidderFirstPartyDataToRequest(request) {
  	const bidderConfig = config.getBidderConfig();
  	const fpdConfigs = Object.keys(bidderConfig).reduce((acc, bidder) => {
  	const currBidderConfig = bidderConfig[bidder];
  	if (currBidderConfig.ortb2) {
  	const ortb2 = mergeDeep({}, currBidderConfig.ortb2);
  	acc.push({
  	bidders: [ bidder ],
  	config: { ortb2 }
  	});
  	}
  	return acc;
  	}, []);
  	if (fpdConfigs.length) {
  	deepSetValue(request, 'ext.prebid.bidderconfig', fpdConfigs);
  	}
  	}

[dgirardi]

I am now thinking this should be a 7.0 change: my proposal would "freeze" the contents of getConfig('ortb2') for a particular auction at the time one calls requestBids, so if a publisher is currently updating it after that point (for example from custom event handlers) we are likely to break their logic.

In fact it will probably have more side effects: currently RTD modules are using auction information to update the global 'ortb2', but if you have multiple simultaneous auctions, there's in general no guarantee on the order they'll run in, so I cannot replicate the same behavior for auction-specific FPD.

@patmmccann, do you have a sense of how publishers are currently working around this? Are they likely to do something like pbjs.onEvent('BID_REQUESTED', () => config.setConfig('ortb2', ...))?

[bretg]

Thanks for this proposal @dgirardi .

I would like to question the need for bidder-specific AND auction-specific ortb2 data. The invariant originally envisioned is that everything underneath ortb2 is actually ORTB. Adding a layer underneath that (standard, BIDDER) muddies that model. I propose that we skip bidder-specific ortb2 for the first release. If we're forced to add it by a real use case in the future, we can follow this design pattern and add a new thing like 'bidder-ortb2' as a peer to 'ortb2' in the auction call.

A clarification on the PBS bid adapter piece:

• the merging of ortb2 into the PBS JSON would be the getConfig({ortb2}) and the bidderRequest.ortb2.
• in the future, if we have to support bidderRequest.bidder-ortb2.BIDDERA, that would need to get sent to PBS merged into ext.prebid.bidderconfig.BIDDERA.ortb2.

Finally, it would nice for the RTD modules to support some kind of merge function to simplify their processing of bidderRequest.ortb2. Or maybe that's doable in native Javascript?

[dgirardi]

@bretg the bidder and auction specific FPD would not be part of the publisher API, rather a simplification of how the RTD modules set bidder-specific data. Currently the flow is:

1. publisher calls requestBids(...)
2. RTD modules "intercept" requestBids, look at its parameters (the adUnits, mostly) and call setConfig({ortb2: ...}) and setBidderConfig(config: {ortb2: ...})
3. bid adapters finally call getConfig('ortb2') which automagically merges global and bidder configs.

The proposal here is to change it to:

1. publisher calls requestBids({ortb2: ...})
2. internally, that calls some other (private) entry point, let's call it startAuction({ortb2: {standard: ..., bidderA: ...}}) (where the difference is FPD is scoped by bidder - and also implicitly by auction, since we are inside a particular one)
3. RTD modules intercept the new startAuction, and just modify the ortb2 argument rather than setting global or bidder configuration
4. bid adapters get the merged ortb2 in the request.

The reason I'd prefer this is that RTD modules are effectively already working at that level of granularity - they run on each auction, and they set bidder-specific FPD. I think right now there's an implicit assumption that every auction will always result in the same set of FPD, for all bidders, by all RTD modules - if that's not true, the result of calling getConfig('ortb2') from an adapter is not necessarily correct. If we change the interface so that RTD just accepts and returns auction and bidder specific FPD this problem goes away completely.

[dgirardi]

The problem remains, however, that in general I don't think it's possible to introduce auction-level FPD without breaking compatibility with publisher's custom code that may be modifying the global ortb2 config - which is likely to include any current workaround to this issue.

To clarify, the problem is that if I call requestBids(ortb2: auctionFPD), it's nigh impossible to make it so that custom code calling getConfig('ortb2' ) will reliably see it reflect auctionFPD when it should; likewise, if the custom code calls setConfig({ortb2: ...}), we cannot guarantee that we'll attach it to the right auction.

[bretg]

I think this all sounds ok, and agree it's an improvement over what's there now. And that it's best to do this in 7.0 so it can be documented for pubs to be aware of.

What I'm reacting to is the aspect that children of ortb2 that aren't actually openrtb. I don't like these: ortb2.standard and ortb2.bidderA . Everything else I'm ok with.

I think what you're saying is that the external interface would still adhere to ortb2 but that the private entry point (e.g. startAuction) splits them. I can see why that could work, but would suggest the alternate startAuction({ortb2: {}, bidderOrtb2:{ bidderA: ...}}) -- this is to keep the invariant that the contents of ortb2 are always actual openrtb.

[patmmccann]

@patmmccann, do you have a sense of how publishers are currently working around this? Are they likely to do something like pbjs.onEvent('BID_REQUESTED', () => config.setConfig('ortb2', ...))?

I think the current workflow for someone who cares deeply about this is to run all your display and then do a new setconfig and then call video. However, in most cases I think video data are just getting appended to display requests incorrectly.

[bretg]

Discussed with Patrick. We approve your proposal @dgirardi and are ready to see it happen as part of 7.0.

We also agree that the the bidder-AND-auction specific scenario can wait for a later phase.