Proposal - expand "extra" bid support

[bretg]

Type of issue

enhancement

Description

Related to:

• Optionally avoid multiple n/w calls when original bidder and alias bidder configured together. #8074
• Support for Prebid Server Bid Adapter to support 'extra' bids #7688

There have been several use cases come up lately where bid adapters are multiplexing multiple demand sources behind the scenes. This is beyond the concept of "alias" in that the PBJS AdUnit may not contain the biddercode at all. Certainly in the case of the long-dreamed Prebid.less scenario, there wouldn't be a list of bidders defined client-side.

In some scenarios, this behavior warrants caution, as it could result in "bid jamming" -- a DSP seeing a given auction more than once. However, there are valid scenarios, so this issue proposes conventions and controls.

Conventions

We propose that two fields can manage this scenario:

• bidResponse.bidderCode indicates the entity making the bid. This is already set by bid adapters that support aliases and by the recent pbsBidAdapter changes allowing "extra" bids. (e.g. districtm)
• bidResponse.adapterCode indicates the bidder code that was in the adunit. This might be an alias of an underlying bid adapter. e.g. it would be "groupm" and not "pubmatic". This is a new field that would be set to the label used by the adunit.

Controls

The PBS "extra" bids feature established a flag to let publishers decide whether bidders not declared in the AdUnit can bid.

pbjs.setConfig({
   s2sConfig: {
       ...
       allowUnknownBidderCodes: true
   }
});

This flag defaults to false.

Proposals

1. extend this control to client-side bidders on a per-adapter basis and add the ability to control which biddercodes may be added:

pbjs.bidderSettings = {
    standard: {
         allowAlternateBidderCodes: false,    // the default if not specified for a specific adapter
         [...]
    },
    bidderA: {
        allowAlternateBidderCodes: true,
        allowedAlternateBidderCodes: ["bidderX", "bidderY"], // bidderA may add X or Y if it wishes
        [...]
    },
    bidderB: {
        allowAlternateBidderCodes: true,
        allowedAlternateBidderCodes: ["*"],        // the default allows this bidder to respond with any biddercodes
        [...]
    }
}

1. Update PBJS-core to enforce these rules: if the bid response bidderCode is different than the adapterCode:
   1. check bidderSettings.adapterCode.allowAlternateBidderCodes. If it exists:
      1. If false, reject
      2. if true, then check allowedAlternateBidderCodes.
         1. If doesn't exist or ["*"], then allow.
         2. Else, check that bidderCode is on the allowedAlternateBidderCodes list. If true, allow. If false, reject.
   2. else, check bidderSettings.standard.allowAlternateBidderCodes. If it exists, do the same as above.
   3. If neither exists, in 6.X, assume 'true' and allow. In 7.0, assume 'false' and reject.
2. Update the general processing of bidderSettings (for cpm adjustment, targeting, etc). If bidderCode doesn't exist, check for a block corresponding to adapterCode.

If allowAlternateBidderCodes is false for a given adapterCode, they may only bid for the biddercode that was in the adunit. Note: this is a breaking change because PBJS is not currently enforcing this rule.

If allowAlternateBidderCodes is true for a given adapterCode and no allowedAlternateBidderCodes is specifed, that adapter may bid for whatever bidderCode it wishes.

We assume that publisher-defined aliases are also set in the adunit, so no special treatment is needed.

Impacts

Because the "bidderCode" in PBJS has always meant "demand source", this change is mostly invisible to PBJS-core and other modules.

However, all of the enforcements in bidderSettings should be considered to be defined for BOTH the bidderCode and the adapterCode. i.e. everywhere one of the settings is enforced, PBJS core would need to check first for a matching bidderCode, and then for a matching adapterCode.

e.g.

pbjs.bidderSettings = {
    standard: {
         allowAlternateBidderCodes: false,    // the default if not specified for a specific adapter
         [...]
    },
    bidderA: {
        bidCpmAdjustment: function(bidCpm, bid){ return bidCpm * .90; },
        allowZeroCpmBids: true
    },
    adapterB: {
        allowAlternateBidderCodes: true,
        bidCpmAdjustment: function(bidCpm, bid){ return bidCpm * .85; },
        allowZeroCpmBids: false
    }
}

In this config, adapterB is the tech platform. Say that adapter returns a bid response object declaring bidderA. It is allowed to do this because allowAlternateBidderCodes: true.

When PBJS-core looks up the bidCpmAdjustment and allowZeroCpmBids options, it first looks for bidderSettings.bidderA. In this case it finds them, so utilizes those values. If they weren't present, then it would look for 'adapterB' and utilize those values.

Reporting

It would be up to analytics adapters to utilize adapterCode if they wanted to be able to aggregate reporting.

Test Scenarios

pbjs.bidderSettings = {
    standard: {
         allowAlternateBidderCodes: false,    // the default if not specified for a specific adapter
         [...]
    },
    bidderA: {
        allowAlternateBidderCodes: true,
        allowedAlternateBidderCodes: ["bidderX", "bidderY"]
    },
    bidderB: {
        allowAlternateBidderCodes: true,
        ...
    }
    bidderC: {
        ...
    }
}

pbjs.aliasBidder('bidderA', 'bidderAA');
pbjs.aliasBidder('bidderB', 'bidderBB');

AdUnit Contains: bidderA
BidResponse Contains: bidderA
Allowed? yes
Why? because a bidder can always bid for itself

AdUnit Contains: bidderA
BidResponse Contains: bidderX
Allowed? yes
Why? bidderSettings.bidderA.allowAlternateBidderCodes is true and bidderSettings.bidderA.allowedAlternateBidderCodes contains bidderX

AdUnit Contains: bidderAA
BidResponse Contains: bidderA
Allowed? No
Why? Even though bidderAA is defined as an alias of bidderA, it's not on bidderSettings.bidderA.allowedAlternateBidderCodes

AdUnit Contains: bidderC
BidResponse Contains: bidderJ
Allowed? No
Why? bidderSettings.bidderC does not specify allowAlternateBidderCodes. Standard.allowAlternateBidderCodes is false, so bidderC is not allowed to bid on behalf of anyone else.

AdUnit Contains: bidderBB
BidResponse Contains: bidderZ
Allowed? no
Why? Same as the above example - aliases don't matter here. bidderSettings is not defined for bidderBB, so it falls back to bidderSettings.standard.allowAlternateBidderCodes, which is false.

[bretg]

Added the ability to control which biddercodes are added by which bidders. The allowedUnknownBidderCodes property of bidderSettings.

[patmmccann]

https://github.com/prebid/Prebid.js/pull/8193/files appears to be an example of this

[pm-azhar-mulla]

Hi @bretg @patmmccann ,
I am from PubMatic and started the development for this draft. I will raise the PR shortly. You can share your thoughts on it, we can take it forward from there.

[bretg]

Conversation copied from the PR...

Aliases are declared valid names that should be valid for bid adapters. This feature is about unexpected names in the response.

So in the following example, both scenarios are valid:

pbjs.aliasBidder('pubmatic', 'foobar');
pbjs.bidderSettings = {
     pubmatic: {
        allowUnknownBidderCodes: true,
        allowedUnknownBidderCodes: ['unexpected']
     }
}
pbjs.requestBids({
   adUnits: [{
       code: 'div-1'
       bids: [{
             bidder: 'foobar'    // the pubmatic adapter decides to set `bidResponse.bidderCode = 'groupm'`
        }]
   }, {
       code: 'div-2',
       bids: [{
              bidder: 'groupm'  // the groupm (pubmatic) adapter decides to set `bidResponse.biddercode = 'pubmatic'`
       }]
   }]
})

For div-1, PBJS calls the pubmatic adapter due to the dynamic alias from foobar. Pubmatic sets "groupm" as the biddercode, which is considered ok because the adapter declares the alias.

For div-2, PBJS calls the pubmatic adapter due to the hard-coded alias groupm. Pubmatic sets "groupm" as the biddercode, which is considered ok because that's the adapter name.

Pubmatic could also choose to return "foobar" as a biddercode because the page set that up as a dynamic alias. Pubmatic could also choose to return "unexpected" as a biddercode because the page specifically set that up as an allowed code.

Here are some things pubmatic couldn't return as a biddercode without being rejected: rubicon, unknown, avengers.

So PBJS-core should allow the following as valid biddercodes:

the actual biddercode
hard-coded aliases
page-defined aliases
bidderSettings.BIDDER.allowedUnknownBidderCodes

[patmmccann]

Would bid adjustment functions on 'pubmatic' apply to 'groupm' responses in the response above @pm-harshad-mane ?

It seems we should fallback(?) to the adapter code bid adjustment function if the bidder code bid adjustment is not defined? It seems this behavior might be considered breaking? It could be behind a flag and default to false to avoid the breaking change. It could be a global flag, or if there is demand, it could be an adapter flag?

[dgirardi]

To clarify, the question above is about whether configuration bidderSettings should fall back to the adapter code (not necessarily just for cpm adjustment).

[pm-azhar-mulla]

@patmmccann @dgirardi I think, it should fallback to adapter code's bidderSettings. I have made the related changes for other bidderSettings property as well such as allowZeroCpmBids, bidCpmAdjustment etc. If the core PR looks good, i can commit the other changes, so you can have a look at it too.

[pm-azhar-mulla]

For now, we have kept the fallback logic to the adapter's bidderSetting.
Scenario 1
Only adapter level settings

pbjs.bidderSettings = {
    standard: {
         allowUnknownBidderCodes: false,   
    },
    pubmatic: {
        allowUnknownBidderCodes: true,
        bidCpmAdjustment: function(bidCpm, bid){ return bidCpm * .85; },
        allowZeroCpmBids: false
    }
}

If pubmatic bids for any unknown bidder, all bidders for which pubmatic bids will have settings added for pubmatic

Scenario 2
Both bidder level and adapter level settings

pbjs.bidderSettings = {
    standard: {
         allowUnknownBidderCodes: false,   
    },
     unexpectedBidder:{
         bidCpmAdjustment: function(bidCpm, bid){ return bidCpm * .5; },
     }
    pubmatic: {
        allowUnknownBidderCodes: true,
        bidCpmAdjustment: function(bidCpm, bid){ return bidCpm * .85; },
        allowZeroCpmBids: false
    }
}

If pubmatic bids for any unknown bidders

• bid with bidderCode as unexpectedBidder would apply settings mentioned in bidderSettings object for unexpectedBidder.
• bid with any other bidderCode would apply settings mentioned for pubmatic

Proposal
@patmmccann @dgirardi I had a discussion with @pm-harshad-mane. Summarising the discussion below.

Now if we maintain a flag at the adapter level it will add to the complexity since some bidders of the adapter might want to overwrite the settings and some won't or some of the bidders may opt-out of the settings completely. We might need to have a flag at the bidder level.
But since we are already supporting bidder alias functionality and never faced this issue, we are of the opinion that we can take this flag based bidderSettings approach in future and stick to the current approach of fallback to the adapter's setting for now.
Please share your views on this.

[dgirardi]

@pm-azhar-mulla, the current approach only falls back to standard. With the current version of #8216, some of the settings will resolve aliases - which is different from what we're discussing here. (That is, requests for groupm will fall back to pubmatic for some of the settings - nothing changes on the response side).

I don't know what makes sense - especially if a bid from the same SSP can come in through two different adapters. If I have:

pbjs.bidderSettings = {
    bidderA: {
         allowUnknownBidderCodes: true,
         allowedUnknownBidderCodes: ['*'],
         bidCpmAdjustment: (cpm) => cpm * 0.5
    },
    bidderB: {
         allowUnknownBidderCodes: true,
         allowedUnknownBidderCodes: ['*'],
         bidCpmAdjustment: (cpm) => cpm * 0.8
    },
}

and bidderA replies with a bid from bidderB and vice-versa, which cpm adjustments should we apply?

[bretg]

Made updates to the proposal above to flesh out details

• As @dgirardi asks, which cpm adjustments should apply
• renamed to allowAlternateBidderCodes
• clarified the role of aliases
• added test cases

[pm-azhar-mulla]

Hi @bretg,
Should I start the work for these updates?

[bretg]

@pm-azhar-mulla - that would be great. No more (planned) changes.

[pm-azhar-mulla]

HI @bretg,

I have a couple of questions about the proposal

1. What should be the fallback mechanism for bidders

Why I am asking this question?

• In the last test case, we have a bidder bidderBB who is an alias of bidderB. Since bidderBB doesn't have bidderSettings of its own, it falls back to bidderSettings of its underlying bid adapter bidderB and allows the bid.
• But in the proposal under 2.ii it is mentioned that if bidderSettings is not present for the given adapterCode, we should fallback to standard bidderSetting

I am raising this point just to get clarity. Can you please help me in understanding which fallback mechanism should we consider for allowAlternateBidderCodes ?

2. Which cpm adjustments should apply (existing bidder settings fallback)

Currently, we are not supporting fall back mechanism for alias, if we make such a change to the core, current publishers who are using alias would break.
e.g: (Normal bidder alias case)

pbjs.bidderSettings = {
    bidderB: {
         bidCpmAdjustment: (cpm) => cpm * 0.8
    }
}
pbjs.aliasBidder('bidderB', 'bidderBB');

If we implement the fallback logic for cpmAdjustment, bidderBB would start to apply cpm adjustment of bidderB to bidderBB, where the publisher doesn't want that. This could be breaking change.
I propose to use the new fallback logic for all bidderSettings in 7.0 and continue to use it as-is for 6.X.

Workaround
If any publisher wants to set bidderSettings for an alternate bidder, they can add it explicitly in the bidderSettings

I hope this makes sense. Please let me know your thoughts, so I can proceed accordingly.

[bretg]

1. What should be the fallback mechanism for bidders

You're right - the last example was wrong. Aliases don't matter - there's no fallback from aliases. Fixed the example.

For the record, I checked the documentation for biddersettings and there's no mention of aliases in the current feature. I don't believe the code supports an alias fallback either, and your second question seems to confirm this. The only currently implemented fallback there is bidderCode-to-standard.

2. Which cpm adjustments should apply (existing bidder settings fallback)

Agreed - adding alias fallback to this feature was not intended. Thanks for catching it.

[dgirardi]

My understanding of bidderSettings fallback is: for bids where the adUnit requests bids: [{bidder: 'bidderA'}], but the response comes back with bidderCode: 'bidderB' (and bidderB is allowed as an alternate code for bidderA or 'standard'), we check in order: bidderB, then bidderA, then 'standard'.

In the terminology of this proposal that would be bidderCode, then adapterCode, then 'standard' (I'm trying to avoid confusion because adapterCode took a different meaning in #8216). There is no longer any consideration of aliases for any bidderSettings lookup.

[bretg]

The algorithm noted above only covers the scenario where bidderCode (from the bid) and adapterCode (what was in the AdUnit) are different... the "unexpected" scenario. In that world, we didn't think 'bidderCode' would be in bidderSettings... it's unexpected, after all.

[patmmccann]

prebid/prebid-server#2174 appears to solve for "Update the pbsBidAdapter to look in bidderSettings as well as s2sConfig and enforce allowedAlternateBidderCodes." Publishers do not need client side functionality to define this per request.

@pm-harshad-mane how does this seem to you?

[pm-harshad-mane]

Sounds good to me @patmmccann !

[bretg]

We discussed in committee and propose that PBJS will not enforce the allowAlternateBidderCodes setting for server-based requests at this point because there's not currently a way to receive both 'bidderCode' and 'adapterCode' until Prebid Server issue prebid/prebid-server#2174 is implemented.

This means that for now, the existing 'allowUnknownBidderCodes' is how pubs can validate/enforce what comes back from PBS. In the future, once PBS reports both seatbid.seat and seatbid.bid.ext.prebid.adapterCode, the PBS bid adapter could implement the allowAlternateBidderCodes logic.