Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GDPR consentManagement module #2213

Merged
merged 52 commits into from
May 1, 2018
Merged
Show file tree
Hide file tree
Changes from 25 commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
bcf27d8
initial commit
jsnellbaker Feb 14, 2018
cda55fe
wip update 2
jsnellbaker Feb 15, 2018
77de2e5
wip update 3
jsnellbaker Feb 15, 2018
ebe1ee8
example
mkendall07 Feb 15, 2018
3cda142
clean up
mkendall07 Feb 15, 2018
6337a2e
wip update 3
jsnellbaker Feb 16, 2018
78f0e9c
hook setup for callBids
jsnellbaker Feb 16, 2018
2412b4f
wip update 4
jsnellbaker Feb 20, 2018
b8c1da6
changed gdpr code to be async-like
jsnellbaker Feb 20, 2018
173d945
cleaned up the callback chain
mkendall07 Feb 20, 2018
6f57a89
added iab cmp detection logic
jsnellbaker Feb 20, 2018
83a4ed9
Merge branch 'master' into gdpr
jsnellbaker Feb 21, 2018
2a587eb
moved hook, reverted unit test changes, and restructed gdpr module
jsnellbaker Feb 23, 2018
2018378
renaming module from gdpr to consentManagement
jsnellbaker Feb 26, 2018
0081610
prebidserver adatper update, additional changes in module
jsnellbaker Feb 28, 2018
a46ca14
updated unit tests for all areas, updates to module logic and structu…
jsnellbaker Mar 1, 2018
6739fd0
adding missing default value
jsnellbaker Mar 6, 2018
63d0d21
removing accidentally committed load time testing code
jsnellbaker Mar 6, 2018
effc19c
changes to layout of consentManagement code and other items based on …
jsnellbaker Mar 9, 2018
e6d8068
moved unit test to different location
jsnellbaker Mar 9, 2018
326e712
finished incomplete unit test in appnexusBidAdapter_spec file
jsnellbaker Mar 9, 2018
112a61b
altered CMP function call logic
jsnellbaker Mar 13, 2018
91b6d83
refactored consentManagement AN lookup function and added gdprDataHan…
jsnellbaker Mar 19, 2018
a425228
some minor cleanup from previous commit
jsnellbaker Mar 19, 2018
f273018
change spacing to try to fix travis issue
jsnellbaker Mar 19, 2018
2e465fd
added scenario to support consentTimeout=0 skip setTimeout
jsnellbaker Mar 22, 2018
540b4b5
updated some comments
jsnellbaker Mar 23, 2018
4de3df0
refactored exit logic for module
jsnellbaker Mar 23, 2018
7f78734
added support for consentRequired field in config
jsnellbaker Mar 27, 2018
d6a4807
merge master into gdpr; fixed conflicts
jsnellbaker Mar 28, 2018
8d23307
remove internal consentRequired default
jsnellbaker Mar 29, 2018
2ccfedf
minor comment fixes
jsnellbaker Apr 2, 2018
a96b129
comment fixes that should be have part of last commit
jsnellbaker Apr 2, 2018
b7811f8
Merge branch 'master' into gdpr
jsnellbaker Apr 12, 2018
9a1f09b
fix includes issue and added gdprConsent to getUserSyncs function
jsnellbaker Apr 12, 2018
73d02f0
Merge branch 'master' into gdpr
jsnellbaker Apr 12, 2018
5ae5eff
renamed default CMP and config field to cmpApi
jsnellbaker Apr 12, 2018
78fcc64
wip - using postmessage to call cmp
jsnellbaker Apr 13, 2018
c39b12d
postMessage workflow added, removed CMP eventlistener check
jsnellbaker Apr 13, 2018
213f2fa
removed if statement
jsnellbaker Apr 13, 2018
405a6f2
cleanup; removed variable and unneeded comments
jsnellbaker Apr 16, 2018
fc41a5a
add gdpr tests pages
jsnellbaker Apr 18, 2018
296e9ca
merge 'master' into branch 'gdpr' + resolve conflict
jsnellbaker Apr 18, 2018
750797a
updates for 1.1 CMP spec
jsnellbaker Apr 24, 2018
effa5b5
remove rogue debugger in unit test
jsnellbaker Apr 24, 2018
a3ca63f
restructured 1.1 CMP iframe code, renamed utils function, cleaned up …
jsnellbaker Apr 26, 2018
4bc0f9b
GDPR support in adform adapter (#2396)
Pupis Apr 27, 2018
4a6e273
merge branch 'master' into branch 'gdpr' + resolve conflicts
jsnellbaker Apr 27, 2018
2f32574
Add gdpr support for PubMaticBidAdapter (#2469)
PubMatic-OpenWrap Apr 30, 2018
f042d7c
GDPR support for AOL adapter (#2443)
vzhukovsky Apr 30, 2018
56f6df4
removing iframe example pages
jsnellbaker Apr 30, 2018
1a35235
comment updates
jsnellbaker Apr 30, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions modules/appnexusBidAdapter.js
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,14 @@ export const spec = {
if (member > 0) {
payload.member_id = member;
}
if (bidderRequest && bidderRequest.gdprConsent) {
// note - objects for impbus use underscore instead of camelCase
payload.gdpr_consent = {
consent_string: bidderRequest.gdprConsent.consentString,
consent_required: bidderRequest.gdprConsent.consentRequired
};
}

const payloadString = JSON.stringify(payload);
return {
method: 'POST',
Expand Down
164 changes: 164 additions & 0 deletions modules/consentManagement.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,164 @@
/**
* This module adds GDPR consentManagement support to prebid.js. It interacts with
* supported CMPs (Consent Management Platforms) to grab the user's consent information
* and make it available for any GDPR supported adapters to read/pass this information to
* their system.
*/
import * as utils from 'src/utils';
import { config } from 'src/config';
import { gdprDataHandler } from 'src/adaptermanager';

const DEFAULT_CMP = 'appnexus';
const DEFAULT_CONSENT_TIMEOUT = 10000;
const DEFAULT_ALLOW_AUCTION_WO_CONSENT = true;

export let userCMP;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think these vars need to be exported?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm exporting these vars so I can read them in the consentManagement_spec.js test file for the setConfig unit tests.

export let consentTimeout;
export let allowAuction;

let consentData;

let context;
let args;
let nextFn;

let timer;
let haveExited;

// add new CMPs here, with their dedicated lookup function that passes the consentString to postLookup()
const cmpCallMap = {
'appnexus': lookupAppNexusConsent
};

/**
* This function handles interacting with the AppNexus CMP to obtain the consentString value of the user.
* Given the asynch nature of the CMP's API, we pass in acting success/error callback functions to exit this function
* based on the appropriate result.
* @param {function(string)} cmpSuccess acts as a success callback when CMP returns a value; pass along consentString (string) from CMP
* @param {function(string)} cmpError acts as an error callback while interacting with CMP; pass along an error message (string)
*/
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This API is much better 👍

function lookupAppNexusConsent(cmpSuccess, cmpError) {
if (!window.__cmp) {
return cmpError('AppNexus CMP not detected.');
}

// first lookup - to determine if new or existing user
// if new user, then wait for user to make a choice and then run postLookup method
// if existing user, then skip to postLookup method
window.__cmp('getConsentData', 'vendorConsents', function(consentString) {
if (consentString == null) {
window.__cmp('addEventListener', 'onSubmit', function() {
// redo lookup to find new string based on user's choices
window.__cmp('getConsentData', 'vendorConsents', cmpSuccess);
});
} else {
cmpSuccess(consentString);
}
});
}

/**
* If consentManagement module is enabled (ie included in setConfig), this hook function will attempt to fetch the
* user's encoded consent string from the supported CMP. Once obtained, the module will store this
* data as part of a gdprConsent object and transferred to adaptermanager's gdprDataHandler object.
* This information is later added into the bidRequest object for any supported adapters to read/pass along to their system.
* @param {object} config This is the same param that's used in pbjs.requestBids. The config.adunits will be updated.
* @param {function} fn The next function in the chain, used by hook.js
*/
export function requestBidsHook(config, fn) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would like to see JSdocs notation on exported functions.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added some more descriptive information to the exported functions.

context = this;
args = arguments;
nextFn = fn;
haveExited = false;

// in case we already have consent (eg during bid refresh)
if (consentData) {
return nextFn.apply(context, args);
}

if (!Object.keys(cmpCallMap).includes(userCMP)) {
utils.logWarn(`CMP framework (${userCMP}) is not a supported framework. Aborting consentManagement module and resuming auction.`);
return nextFn.apply(context, args);
}

// lookup times and user interaction with CMP prompts can greatly vary, so enforcing a timeout on the CMP process
timer = setTimeout(cmpTimedOut, consentTimeout);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please skip the setTimeout if consentTimeout is 0. Otherwise we lose out place in the javascript stack.

Copy link
Collaborator Author

@jsnellbaker jsnellbaker Mar 22, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mkendall07 talked with me about this change this morning. I've got the changes put together and will include them in the next commit.

As a confirmation question - if the publisher were to have the setting to cancel the auction enabled, the combination of consentTimeout=0 + the cancel setting would cause a new user's auction to be canceled immediately during that first page load where they have to fill out the consent information.

Any subsequent page loads for that new user would go through the normal workflow and should be fine (even if the publisher had the consentTimeout at 0, but is that first time canceled auction an issue? Or is it fine to leave as is?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First time cancelation is fine -- it's a pretty straightforward implication of the publisher's wishes in this scenario, IMO.


cmpCallMap[userCMP].call(this, processCmpData, exitFailedCmp);
}

// after we have grabbed ideal ID from CMP, apply the data to adUnits object and finish up the module
function processCmpData(consentString) {
if (typeof consentString !== 'string' || consentString === '') {
exitFailedCmp(`CMP returned unexpected value during lookup process; returned value was (${consentString}).`);
}
clearTimeout(timer);

// to stop the auction from running if we chose to cancel and timeout was reached
if (haveExited === false) {
storeConsentData(consentString);
nextFn.apply(context, args);
}
}

// store CMP string in module and invoke gdprDataHandler.setConsentData() to make information available in adaptermanger.js
function storeConsentData(cmpConsentString) {
consentData = {
consentString: cmpConsentString,
consentRequired: true
};
gdprDataHandler.setConsentData(consentData);
}

function cmpTimedOut() {
exitFailedCmp('CMP workflow exceeded timeout threshold.');
}

// controls the exit of the module based on consentManagement config; either we'll resume the auction or cancel the auction
function exitFailedCmp(message) {
clearTimeout(timer);
haveExited = true;
if (allowAuction) {
utils.logWarn(message + ' Resuming auction without consent data as per consentManagement config.');
storeConsentData(undefined);

nextFn.apply(context, args);
} else {
utils.logError(message + ' Canceling auction as per consentManagement config.');
}
}

/** Simply resets the module's consentData variable back to undefined */
export function resetConsentData() {
consentData = undefined;
}

/**
* A configuration function that initializes some module variables, as well as add a hook into the requestBids function
* @param {object} config consentManagement module config settings; cmp (string), timeout (int), allowAuctionWithoutConsent (boolean)
*/
export function setConfig(config) {
if (typeof config.cmp === 'string') {
userCMP = config.cmp;
} else {
userCMP = DEFAULT_CMP;
utils.logInfo(`consentManagement config did not specify cmp. Using system default setting (${userCMP}).`);
}

if (typeof config.timeout === 'number') {
consentTimeout = config.timeout;
} else {
consentTimeout = DEFAULT_CONSENT_TIMEOUT;
utils.logInfo(`consentManagement config did not specify timeout. Using system default setting (${consentTimeout}).`);
}

if (typeof config.allowAuctionWithoutConsent !== 'undefined') {
allowAuction = config.allowAuctionWithoutConsent;
} else {
allowAuction = DEFAULT_ALLOW_AUCTION_WO_CONSENT;
utils.logInfo(`consentManagement config did not specify allowAuctionWithoutConsent. Using system default setting (${allowAuction}).`);
}

$$PREBID_GLOBAL$$.requestBids.addHook(requestBidsHook, 50);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I always forget what the addHook signature looks like. What's the 50 value signify?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See comment above for the change in the pre1api file.

}
config.getConfig('consentManagement', config => setConfig(config.consentManagement));
2 changes: 1 addition & 1 deletion modules/pre1api.js
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,7 @@ pbjs.requestBids.addHook((config, next = config) => {
} else {
logWarn(`${MODULE_NAME} module: concurrency has been disabled and "$$PREBID_GLOBAL$$.requestBids" call was queued`);
}
}, 100);
}, 5);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's the impact of this change?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The number (also relates to the other comment below) acts as a priority level for the hooked functions (when there are multiple hooked functions on the same hook).

The pbjs.requestBids function has 3 hooked functions from 3 different pbjs modules: pre1api, PubCommonId and consentManagement. So lowering this priority level ensures that the pre1api module's hooked function will (generally) go last in the sequence if all these modules were enabled together.

Having the pre1api module go last (specifically after consentManagement) is related to the need for the pre1api to execute right before the pbjs.requestBids so it knows which auction is currently active. The consentManagement module uses/waits on callbacks (to retrieve information from the CMP), and this buffering of the auction process could cause issues for the pre1api knowing which auction is active if there was a time gap because it ran first. So the lower priority helps avoid this scenario.

The 5 specifically is lower than the default priority that's set for any hooked function (which is 10), but still higher than the base function pbjs.requestBids (which is 0). So it should generally always the be last hook to run before the pbjs.requestBids would execute.


Object.keys(auctionPropMap).forEach(prop => {
if (prop === 'allBidsAvailable') {
Expand Down
11 changes: 8 additions & 3 deletions modules/prebidServerBidAdapter.js
Original file line number Diff line number Diff line change
Expand Up @@ -251,7 +251,7 @@ function _getDigiTrustQueryParams() {
*/
const LEGACY_PROTOCOL = {

buildRequest(s2sBidRequest, adUnits) {
buildRequest(s2sBidRequest, bidRequests, adUnits) {
// pbs expects an ad_unit.video attribute if the imp is video
adUnits.forEach(adUnit => {
const videoMediaType = utils.deepAccess(adUnit, 'mediaTypes.video');
Expand Down Expand Up @@ -381,7 +381,7 @@ const OPEN_RTB_PROTOCOL = {

bidMap: {},

buildRequest(s2sBidRequest, adUnits) {
buildRequest(s2sBidRequest, bidRequests, adUnits) {
let imps = [];

// transform ad unit into array of OpenRTB impression objects
Expand Down Expand Up @@ -456,6 +456,11 @@ const OPEN_RTB_PROTOCOL = {
request.user = { ext: { digitrust: digiTrust } };
}

if (bidRequests && bidRequests[0].gdprConsent) {
request.regs = { ext: { gdpr: bidRequests[0].gdprConsent.consentRequired ? 1 : 0 } };
request.user = { ext: { consent: bidRequests[0].gdprConsent.consentString } };
}

return request;
},

Expand Down Expand Up @@ -556,7 +561,7 @@ export function PrebidServer() {
.reduce(utils.flatten)
.filter(utils.uniques);

const request = protocolAdapter().buildRequest(s2sBidRequest, adUnitsWithSizes);
const request = protocolAdapter().buildRequest(s2sBidRequest, bidRequests, adUnitsWithSizes);
const requestJson = JSON.stringify(request);

ajax(
Expand Down
16 changes: 16 additions & 0 deletions src/adaptermanager.js
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,16 @@ function getAdUnitCopyForClientAdapters(adUnits) {
return adUnitsClientCopy;
}

exports.gdprDataHandler = {
consentData: null,
setConsentData: function(consentInfo) {
this.consentData = consentInfo;
},
getConsentData: function() {
return this.consentData;
}
};

exports.makeBidRequests = function(adUnits, auctionStart, auctionId, cbTimeout, labels) {
let bidRequests = [];

Expand Down Expand Up @@ -211,6 +221,12 @@ exports.makeBidRequests = function(adUnits, auctionStart, auctionId, cbTimeout,
bidRequests.push(bidderRequest);
}
});

if (exports.gdprDataHandler.getConsentData()) {
bidRequests.forEach(bidRequest => {
bidRequest['gdprConsent'] = exports.gdprDataHandler.getConsentData();
});
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if there's a better way to do this....

}
return bidRequests;
};

Expand Down
26 changes: 24 additions & 2 deletions test/spec/modules/appnexusBidAdapter_spec.js
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,7 @@ describe('AppNexusAdapter', () => {
});
});

it('should attache native params to the request', () => {
it('should attach native params to the request', () => {
let bidRequest = Object.assign({},
bidRequests[0],
{
Expand Down Expand Up @@ -290,7 +290,7 @@ describe('AppNexusAdapter', () => {
}]);
});

it('should should add payment rules to the request', () => {
it('should add payment rules to the request', () => {
let bidRequest = Object.assign({},
bidRequests[0],
{
Expand All @@ -306,6 +306,28 @@ describe('AppNexusAdapter', () => {

expect(payload.tags[0].use_pmt_rule).to.equal(true);
});

it('should add gdpr consent information to the request', () => {
let consentString = 'BOJ8RZsOJ8RZsABAB8AAAAAZ+A==';
let bidderRequest = {
'bidderCode': 'appnexus',
'auctionId': '1d1a030790a475',
'bidderRequestId': '22edbae2733bf6',
'timeout': 3000,
'gdprConsent': {
consentString: consentString,
consentRequired: true
}
};
bidderRequest.bids = bidRequests;

const request = spec.buildRequests(bidRequests, bidderRequest);
const payload = JSON.parse(request.data);

expect(payload.gdpr_consent).to.exist;
expect(payload.gdpr_consent.consent_string).to.exist.and.to.equal(consentString);
expect(payload.gdpr_consent.consent_required).to.exist.and.to.be.true;
});
})

describe('interpretResponse', () => {
Expand Down
Loading