Methodology v1.3 - published 2026-07-02
Methodology
How the ConsentMark scanner measures analytics governance, what the grade means, and what it deliberately does not claim. We grade what a site does before consent, not which libraries it loads. Versioned and citable.
Changelog
Older scans link to the methodology version they were scored under. The grade attached to a specific /scan/<id> URL is the grade as it was recorded on that date, under the methodology in force at the time.
- v1.32026-07-02
Behaviour-based grading of Google Consent Mode v2. We grade enforcement behaviour, not artefact presence: a correctly configured gtag.js loader that writes nothing and sends only a cookieless denied ping before consent no longer caps the grade at D. Inconclusive captures are withheld as I rather than recorded as a non-reproducible D.
- The pre-consent storage gate fires on evidenced storage or identifier transmission - a cookie / localStorage / IndexedDB write, a device identifier, or a payload beyond the cookieless gcs=G100 denied ping - not on the mere presence of the gtag.js loader.
- A correct Consent Mode v2 site caps at B, not A: before consent it still sends the visitor's IP address and the page they are on to Google. Top marks (A) are reserved for sites that send nothing to a third-party measurement provider before consent.
- A verified first-party CNAME (server-side GTM) discloses to the site's own endpoint, not Google, and is not capped.
- Inconclusive capture is withheld, never downgraded: when Consent Mode is detected but the scan captured no enforcement proof, the grade is withheld as I (inconclusive) rather than recorded as a flaky D.
- Clean results are described as 'no pre-consent storage or identifiers observed' - never 'compliant'.
- v1.22026-05-26
Precedent corpus expansion to 20 cited DPA enforcement decisions across DPC, ICO, CNIL, EDPB, AEPD and BfDI. New /precedent/ public route with per-case permalinks. 'Cite this scan' / 'Cite this case' citation blocks added.
- Registry expanded from 7 to 20 cases; AEPD (Spain) and BfDI (Germany) added as primary regulators.
- Public /precedent/ corpus route with filterable index, per-case pages, and versioned JSON export at /precedent/registry.v1.json.
- Citation block on /scan/[id] and /precedent/[slug] - BibTeX, APA, Markdown, plain text.
- Methodology page itself is now versioned. /methodology redirects to the latest version. Scan pages link to the version they were scored under.
- v1.12026-05-15
Three-zone narrative (Observed -> Regulatory context -> Precedent) goes live on /scan/[id]. Dual-citation Wayback snapshots on every registry entry.
- Public scan results render Observed / Regulatory / Precedent zones explicitly, never blended.
- Every primary-source URL in the enforcement registry carries a web.archive.org dual-citation.
- Static-vs-runtime methodology note added for Consent Mode v2 detection.
- v1.02026-04-12
Initial published methodology. Reject-state network-leak threshold (absolute + relative gate). Severity ladder caps grade at C, D, or F based on delta size.
- Reject-state leak detected when post-reject requests exceed pre-consent by both 10 absolute and 10% relative.
- Severity ladder: <=25 delta = Moderate (cap C), <=50 = High (cap D), >50 = Critical (cap F).
- Non-operable reject button caps grade at D regardless of leak size.
How we scan
The scanner loads a public website in a real browser (Playwright + Chromium), records every network request and cookie before consent, after accepting consent, and after rejecting consent. The resulting grade (A through F) is a summary of those observations - principally what the site stores and transmits before the visitor has chosen, and whether it keeps making third-party tracking requests after the visitor has rejected.
Consent lifecycle: the scanner identifies the consent management platform (CMP), exercises its accept and reject controls, and observes the delta in network and cookie behaviour across the three phases. CMP detection happens via DOM signatures + window-object sniffing; no proprietary integration is required from the scanned site.
We grade behaviour, not presence
Google Consent Mode v2 requires the gtag.js library to load before consent. The library loads, reads its denied defaults, and self-suppresses. On a correctly configured site the only pre-consent network traffic is a cookieless gcs=G100("denied") ping, which writes nothing to the device. Loading a script is not, by itself, the storage of or access to information on the device that Article 5(3) turns on. We therefore stopped treating the mere presence of the gtag.js loader as a violation, and grade what the site actually does before consent.
We describe a clean result as "no pre-consent storage or identifiers observed"- never "compliant". We assess storage and transmission behaviour in a single automated visit; we do not adjudicate a site's full GDPR posture.
What no longer caps a grade
An identifier-clean gtag.js loader on a Consent-Mode-credible site is informational, like the gtm.js container loader. Its presence alone no longer caps the grade at D.
Teeth kept
Real pre-consent cookies or identifiers, a granted pre-consent ping, or a genuine tracking hit still fire the pre-consent storage gate. Evidenced harm is graded exactly as before.
Why a correct Consent Mode v2 site is B, not A
Even when Consent Mode v2 is set up correctly, the visitor's IP address and the page they are on are sent to Google before they have chosen. Nothing is stored on their device, but those details still leave for a third party. So a correct Consent Mode v2 site is graded B, not A. A is reserved for sites that send nothing to a third-party measurement provider before consent. A site that keeps tagging on its own domain - a verified first-party CNAME, or server-side GTM - keeps that data on its own systems, so it is not capped.
Inconclusive is withheld, never downgraded
The consent-command and denied-ping capture is non-deterministic. When Consent Mode is detected but the scan captured no enforcement proof, the grade is withheld as I (inconclusive) rather than recorded as a non-reproducible D. A top grade rests on positive capture confidence, not on absence of evidence.
What this grade observes - and what it does not
This grade reflects a single unauthenticated pre-consent load. We observe what a site writes to the device (cookies, localStorage) and the identifier keys it sends before consent, in both request URLs and request bodies. We do not currently test for reads of existing identifiers, browser fingerprinting, or the values carried inside request bodies. A clean result means no pre-consent storage and no known identifier keys were observed - not that no data of any kind reached a third party.
Grade movements in v1.3
Re-baselines were human-reviewed, not auto-applied. Our own sites and the prospect fixtures grade as expected, with no silent movement.
| Scenario | Was | Now | Why |
|---|---|---|---|
| Consent Mode v2 gtag site, clean captured denied ping | A | B | The visitor's IP and page are still sent to Google before consent, so it caps at B. |
| Consent Mode v2 gtag site, capture raced (no enforcement proof) | D (flaky) | I | We withhold rather than downgrade on evidence we did not observe. |
| Server-side GTM, verified first-party CNAME collect endpoint | A | A | First-party disclosure, not to Google. Unchanged. |
| Real pre-consent cookie, identifier, or granted ping | D | D | Evidenced pre-consent storage or transmission. Unchanged. |
| Ghost-CMP, post-reject leak, or PII in a tag | - | - | Handled by separate gates (D3 / F4 / F1). Unchanged. |
The reject-state leak threshold
A reject-state leak is recorded only when the post-reject phase fires materially more network requests than the pre-consent phase. Both gates must hold at the same time:
- Absolute floor. At least 10 additional requests after reject compared with the pre-consent phase.
- Relative floor. The additional requests amount to at least 10% of the pre-consent request count.
Worked example
A site fires 86 network requests on first load (pre-consent). After the visitor clicks Reject All, the site goes on to fire 185 requests in total. The delta is 99 - well above the absolute floor of 10 and the relative floor of 10% (99 / 86 = 115%). Both gates clear, so a leak Finding is recorded.
Sites that clear only one gate do not trigger a Finding. The two-gate design is deliberate - it stops single-pixel reload noise on small sites from registering, and it stops heavy-content sites from registering simply because 10-20 background requests is normal post-banner behaviour.
Severity ladder
When a Finding is recorded, the size of the leak determines the maximum grade the site can receive. The ladder is one-way - a Finding can only cap a grade, never raise it.
| Observation | Severity | Grade cap | Reading |
|---|---|---|---|
| delta <= 25 extra requests | Moderate | C | Reject is not clean, but the leak is contained. |
| delta <= 50 extra requests | High | D | Reject produces only token suppression of third-party traffic. |
| delta > 50 extra requests | Critical | F | Reject is effectively inert. Most third-party traffic continues unchanged. |
| Reject button cannot be operated | High | D | A consent platform is detected but no functional reject control could be exercised. |
When more than one Finding fires on the same scan, the strictest cap wins. A critical leak combined with a non-functional reject button caps at F.
Gate definitions
Each grade is the output of a small set of named gates. A scan can clear all the gates (Grade A/B), trip a consent-mechanics gate (Grade C), or trip a substantive storage-or-leak gate (Grade D or F).
- D1. Evidenced pre-consent storage or identifier transmission - a cookie / localStorage / IndexedDB write, a device identifier, a granted pre-consent ping, or a payload beyond the cookieless
gcs=G100denied ping. The presence of thegtag.jsloader alone does not trip D1. - D3. Reject control is detected but cannot be exercised (e.g. missing event handler, asymmetric layer hierarchy, hidden behind a JavaScript-only "Settings" link).
- F-gates. Critical-severity reject leak (delta greater than 50), no detectable consent mechanism at all, or known-broken CMP integration.
- B / C bands. Pre-consent third-party measurement disclosure (a correct Consent Mode v2 site caps at B), consent-mechanics weak points, or excess third-party requests in the consent-accepted state.
- I. Inconclusive. The scanner detected Consent Mode but could not capture enforcement proof within the scan budget, or could not observe a stable consent lifecycle. The grade is withheld rather than awarded or downgraded on absence of evidence.
What this is, and what it isn't
What the grade is
A reproducible, evidence-backed editorial signal. A summary of observable browser behaviour - what a site stores before consent, which trackers fire, when they fire, and whether reject is honoured. Each grade is derived from a versioned set of Findings that the scan emits.
What we deliberately do NOT do
- No AI-generated verdicts. The narrative paraphrases regulator language, never invents legal conclusions.
- No claim of "compliant". A clean grade is "no pre-consent storage or identifiers observed", not a legal conclusion about the site.
- Dual-citation only. Every regulator URL has a web.archive.org Wayback snapshot beside it so a regulator site refactor cannot dead-link the citation.
Why these signals matter
The behaviour-based stance and the reject-state leak threshold are grounded in published regulator guidance and enforcement decisions, not in any private interpretation. The full corpus is published at /precedent/; representative anchors:
- EDPB Cookie Banner Taskforce - Report of work undertaken (Report 2/2023)
European Data Protection Board guidance on the practical application of Article 5(3) of the ePrivacy Directive and the Article 7 GDPR consent obligations to cookie banners and trackers.
- CJEU C-582/14 (Breyer v Germany)
A dynamic IP address is personal data in the hands of a party with the legal means to identify the visitor. The basis for capping a pre-consent Google measurement disclosure at B: the visitor's IP reaches Google before consent.
- CNIL SAN-2020-012 (Google) and SAN-2020-013 (Amazon)
Foundational French DPA cookie precedent: advertising cookies set on arrival without consent are unlawful under Article 82 of the French Data Protection Act.
Disputing a grade
If you believe a published scan misrepresents your site, write to contact@consentmark.com with the scan URL and the specific Finding you contest. We commit to:
- Acknowledging the dispute within two working days.
- A 14-day notification window before any change to the public grade of an affected site, where a regrade is the result of a methodology change rather than a fresh scan.
- Preserving the original scan record alongside any correction, so the audit trail is never silently rewritten.