Methodology v7.6 - published 2026-08-27
Methodology
How the ConsentMark scanner measures analytics governance, what the grade means, and what it deliberately does not claim. We grade what a site does before consent, not which libraries it loads. Versioned and citable.
Scope
Assessments are made exclusively against EU and EEA law - Article 5(3) of Directive 2002/58/EC (the ePrivacy Directive), as amended by Directive 2009/136/EC and transposed into member states' national laws, and the General Data Protection Regulation.
The methodology does not assess the CCPA/CPRA, the LGPD, or any other regime. Sites that do not serve EU or EEA visitors may be subject to different obligations, and a grade recorded here says nothing about them.
Changelog
Older scans link to the methodology version they were scored under. The grade attached to a specific /scan/<id> URL is the grade as it was recorded on that date, under the methodology in force at the time.
14th August 2026 - Scope section added, clarifying the jurisdictional scope of assessment. No change to grading.
- v7.62026-08-27
Renumbering and correction. The document version now matches the scoring-engine version a scan records, so a published grade and the methodology that produced it carry the same number. The gate table is generated from the engine's gate labels rather than written by hand, and two descriptions that did not match the code were removed.
- One version number. This document was v1.3 while every scan recorded methodology 7.6 and the benchmark page cited v1.2. The document version is now derived from the engine constant; /methodology/v1.3 redirects here.
- The gate table is generated at build time from the scoring engine's own gate labels. A gate can no longer be described on this page in words the engine does not use.
- Removed the reject-state leak threshold (an absolute floor of 10 extra requests and a relative floor of 10%) and the delta-sized severity ladder. No such delta test exists in the engine: a single tracking request after reject, or one tracking cookie that survives it, is the finding.
- Withdrew the claim that a top grade depends on a positive capture confirmation. The engine records pre-consent capture confidence as inconclusive, or not at all. It has no confirming state, so the claim described something the code cannot produce.
- v1.32026-07-02
Behaviour-based grading of Google Consent Mode v2. We grade enforcement behaviour, not artefact presence: a correctly configured gtag.js loader that writes nothing and sends only a cookieless denied ping before consent no longer caps the grade at D. Inconclusive captures are withheld as I rather than recorded as a non-reproducible D.
- The pre-consent storage gate fires on evidenced storage or identifier transmission - a cookie / localStorage / IndexedDB write, a device identifier, or a payload beyond the cookieless gcs=G100 denied ping - not on the mere presence of the gtag.js loader.
- A correct Consent Mode v2 site caps at B, not A: before consent it still sends the visitor's IP address and the page they are on to Google. Top marks (A) are reserved for sites that send nothing to a third-party measurement provider before consent.
- A verified first-party CNAME (server-side GTM) discloses to the site's own endpoint, not Google, and is not capped.
- Inconclusive capture is withheld, never downgraded: when Consent Mode is detected but the scan captured no enforcement proof, the grade is withheld as I (inconclusive) rather than recorded as a flaky D.
- Clean results are described as 'no pre-consent storage or identifiers observed' - never 'compliant'.
- v1.22026-05-26
Precedent corpus expansion to 20 cited DPA enforcement decisions across DPC, ICO, CNIL, EDPB, AEPD and BfDI. New /precedent/ public route with per-case permalinks. 'Cite this scan' / 'Cite this case' citation blocks added.
- Registry expanded from 7 to 20 cases; AEPD (Spain) and BfDI (Germany) added as primary regulators.
- Public /precedent/ corpus route with filterable index, per-case pages, and versioned JSON export at /precedent/registry.v1.json.
- Citation block on /scan/[id] and /precedent/[slug] - BibTeX, APA, Markdown, plain text.
- Methodology page itself is now versioned. /methodology redirects to the latest version. Scan pages link to the version they were scored under.
- v1.12026-05-15
Three-zone narrative (Observed -> Regulatory context -> Precedent) goes live on /scan/[id]. Dual-citation Wayback snapshots on every registry entry.
- Public scan results render Observed / Regulatory / Precedent zones explicitly, never blended.
- Every primary-source URL in the enforcement registry carries a web.archive.org dual-citation.
- Static-vs-runtime methodology note added for Consent Mode v2 detection.
- v1.02026-04-12
Initial published methodology. Reject-state network-leak threshold (absolute + relative gate). Severity ladder caps grade at C, D, or F based on delta size.
- Reject-state leak detected when post-reject requests exceed pre-consent by both 10 absolute and 10% relative.
- Severity ladder: <=25 delta = Moderate (cap C), <=50 = High (cap D), >50 = Critical (cap F).
- Non-operable reject button caps grade at D regardless of leak size.
How we scan
The scanner loads a public website in a real browser (Playwright + Chromium), records every network request and cookie before consent, after accepting consent, and after rejecting consent. The resulting grade (A through F) is a summary of those observations - principally what the site stores and transmits before the visitor has chosen, and whether it keeps making third-party tracking requests after the visitor has rejected.
Consent lifecycle: the scanner identifies the consent management platform (CMP), exercises its accept and reject controls, and observes the delta in network and cookie behaviour across the three phases. CMP detection happens via DOM signatures + window-object sniffing; no proprietary integration is required from the scanned site.
We grade behaviour, not presence
Google Consent Mode v2 requires the gtag.js library to load before consent. The library loads, reads its denied defaults, and self-suppresses. On a correctly configured site the only pre-consent network traffic is a cookieless gcs=G100 ("denied") ping, which writes nothing to the device. Loading a script is not, by itself, the storage of or access to information on the device that Article 5(3) turns on. We therefore stopped treating the mere presence of the gtag.js loader as a violation, and grade what the site actually does before consent.
We describe a clean result as "no pre-consent storage or identifiers observed" - never "compliant". We assess storage and transmission behaviour in a single automated visit; we do not adjudicate a site's full GDPR posture.
What no longer caps a grade
An identifier-clean gtag.js loader on a Consent-Mode-credible site is informational, like the gtm.js container loader. Its presence alone no longer caps the grade at D.
Teeth kept
Real pre-consent cookies or identifiers, a granted pre-consent ping, or a genuine tracking hit still fire the pre-consent storage gate. Evidenced harm is graded exactly as before.
Why a correct Consent Mode v2 site is B, not A
Even when Consent Mode v2 is set up correctly, the visitor's IP address and the page they are on are sent to Google before they have chosen. Nothing is stored on their device, but those details still leave for a third party. So a correct Consent Mode v2 site is graded B, not A. A is reserved for sites that send nothing to a third-party measurement provider before consent. A site that keeps tagging on its own domain - a verified first-party CNAME, or server-side GTM - keeps that data on its own systems, so it is not capped.
Inconclusive is withheld, never downgraded
The consent-command and denied-ping capture is non-deterministic. When Consent Mode is detected but the scan captured no enforcement proof, the grade is withheld as I (inconclusive) rather than recorded as a non-reproducible D. The same applies when the pre-consent cookie probe did not complete: the engine marks the capture inconclusive and the grader withholds, because a clean grade would otherwise rest on a look that never happened. The engine records that flag in one direction only - a capture is either marked inconclusive or it is not marked at all. There is no positive confirmation, so a clean grade means nothing disqualifying was observed by a scan that did complete its probes, not that the absence was independently confirmed.
What this grade observes - and what it does not
This grade reflects a single unauthenticated pre-consent load. We observe what a site writes to the device (cookies, localStorage) and the identifier keys it sends before consent, in both request URLs and request bodies. We do not currently test for reads of existing identifiers, browser fingerprinting, or the values carried inside request bodies. A clean result means no pre-consent storage and no known identifier keys were observed - not that no data of any kind reached a third party.
Grade movements in the behaviour-based rebaseline (v1.3)
Re-baselines were human-reviewed, not auto-applied. Our own sites and the prospect fixtures grade as expected, with no silent movement.
| Scenario | Was | Now | Why |
|---|---|---|---|
| Consent Mode v2 gtag site, clean captured denied ping | A | B | The visitor's IP and page are still sent to Google before consent, so it caps at B. |
| Consent Mode v2 gtag site, capture raced (no enforcement proof) | D (flaky) | I | We withhold rather than downgrade on evidence we did not observe. |
| Server-side GTM, verified first-party CNAME collect endpoint | A | A | First-party disclosure, not to Google. Unchanged. |
| Real pre-consent cookie, identifier, or granted ping | D | D | Evidenced pre-consent storage or transmission. Unchanged. |
| Ghost-CMP, post-reject leak, or PII in a tag | - | - | Handled by separate gates (D3 / F4 / F1). Unchanged. |
What rejecting consent has to achieve
There is no threshold and no delta arithmetic. After the scanner operates the reject control, a Finding is recorded if any tracking tag fires, or if a tracking cookie set before the visitor chose is still in the jar afterwards. One request is enough. Earlier versions of this page described an absolute floor of 10 extra requests and a relative floor of 10%, with a severity ladder scaled to the size of the delta. The scoring engine has never implemented that test and the description has been withdrawn.
Three carve-outs exist, all of them in the code rather than in editorial judgement:
- Infrastructure only. If the only traffic after reject is the consent platform and other non-tracking infrastructure, no Finding is recorded.
- Downgraded signal. If Consent Mode v2 is functional, every post-reject tracking request carries an observed on-wire downgrade (
gcsstorage denied, or Microsoftasc=D), and no tracking cookie persisted, the result is the D2 gate rather than the F4 gate. The vendor was told the visitor said no. One unrestricted post-reject tracker removes this carve-out for the whole scan. - Divergent page content. If the accept-branch and reject-branch browsers were served materially different pages (A/B test, geo-targeting, a late SDK), the tag half of the comparison is unsound and is suppressed. Cookie persistence is a direct before-and-after observation of the jar, so it still counts.
Gate definitions
Every grade is the output of a named gate. The letter in the gate code is the cap it applies: an F gate caps the grade at F, a D gate at D, and so on. When more than one gate fires on the same scan the strictest cap wins. A gate can only cap a grade, never raise it.
The table is generated from the scoring engine. These are the exact strings the scanner uses when it names a gate on a result page, so the description here cannot drift from the code that produced your grade.
| Gate | What the scanner observed | Grade cap |
|---|---|---|
| F1 | Personal data found in tracking requests | F |
| F2 | Tracking active with no consent controls | F |
| F3 | Data collection starts before consent | F |
| F4 | Tracking fires on a fresh page load after consent was rejected, or cookies persist | F |
| F5 | Consent signals are contradicting each other | F |
| D1 | Consent platform not blocking all tracking | D |
| D2 | Post-reject tracking continues, but Consent Mode v2 downgrades the signal | D |
| D3 | Consent banner did not render - visitor had no opportunity to consent | D |
| D4 | Tracking already running when consent was rejected finished its in-page work - no new tracking started, and nothing fired on the next page load | D |
| C1 | Rejecting consent is harder than accepting it | C |
| C2 | Data sent outside the EU without documented safeguards | C |
| C3 | Vendor consent signal contradicts user rejection | C |
| B1 | Some tracking before consent, but in restricted mode | B |
| B2 | Tag manager fires before consent, but tracking is gated | B |
| B3 | Self-managing consent tool active before consent platform | B |
| B4 | Non-tracking widgets load pre-consent | B |
| B5 | Google-hosted fonts loaded before consent (third-party data transfer) | B |
- F gates - cap F
- Substantive failure. Tracking without a working consent gate, or personal data in a tracking request.
- D gates - cap D
- Consent infrastructure is present but did not do its job on this visit.
- C gates - cap C
- Consent mechanics or transfer safeguards are weak, but tracking is gated.
- B gates - cap B
- Something reached a third party before consent, in a restricted or non-tracking form.
I - inconclusive, which is not a gate
I is not in the table because nothing was found: it is the grade we publish when the scan could not observe enough to grade at all. The scanner detected Consent Mode but captured no enforcement proof within the scan budget, or a probe it needed did not complete, or it could not observe a stable consent lifecycle. The grade is withheld rather than awarded or downgraded on absence of evidence.
What this is, and what it isn't
What the grade is
A reproducible, evidence-backed editorial signal. A summary of observable browser behaviour - what a site stores before consent, which trackers fire, when they fire, and whether reject is honoured. Each grade is derived from a versioned set of Findings that the scan emits.
What we deliberately do NOT do
- No AI-generated verdicts. The narrative paraphrases regulator language, never invents legal conclusions.
- No claim of "compliant". A clean grade is "no pre-consent storage or identifiers observed", not a legal conclusion about the site.
- Dual-citation only. Every regulator URL has a web.archive.org Wayback snapshot beside it so a regulator site refactor cannot dead-link the citation.
Why these signals matter
The behaviour-based stance and the post-reject gates are grounded in published regulator guidance and enforcement decisions, not in any private interpretation. The full corpus is published at /precedent/; representative anchors:
- EDPB Cookie Banner Taskforce - Report of work undertaken (Report 2/2023)
European Data Protection Board guidance on the practical application of Article 5(3) of the ePrivacy Directive and the Article 7 GDPR consent obligations to cookie banners and trackers.
- CJEU C-582/14 (Breyer v Germany)
A dynamic IP address is personal data in the hands of a party with the legal means to identify the visitor. The basis for capping a pre-consent Google measurement disclosure at B: the visitor's IP reaches Google before consent.
- CNIL SAN-2020-012 (Google) and SAN-2020-013 (Amazon)
Foundational French DPA cookie precedent: advertising cookies set on arrival without consent are unlawful under Article 82 of the French Data Protection Act.
Disputing a grade
If you believe a published scan misrepresents your site, write to contact@consentmark.com with the scan URL and the specific Finding you contest. We commit to:
- Acknowledging the dispute within two working days.
- A 14-day notification window before any change to the public grade of an affected site, where a regrade is the result of a methodology change rather than a fresh scan.
- Preserving the original scan record alongside any correction, so the audit trail is never silently rewritten.