Scoring engine v2
Methodology
How the ConsentMark scanner measures analytics governance, what the grade means, and what it deliberately does not claim.
What ConsentMark measures
The scanner loads a public website in a real browser, records every network request and cookie before consent, after accepting consent, and after rejecting consent. The resulting grade (A through F) is a summary of those observations - principally whether the site keeps making third-party tracking requests after the visitor has rejected. The grade is an editorial signal, not a regulatory verdict. The underlying Findings and request evidence are what matter for any audit or remediation conversation.
The reject-state leak threshold
A reject-state leak is recorded only when the post-reject phase fires materially more network requests than the pre-consent phase. Both gates must hold at the same time:
- Absolute floor. At least 10 additional requests after reject compared with the pre-consent phase.
- Relative floor. The additional requests amount to at least 10% of the pre-consent request count.
Worked example
A site fires 86 network requests on first load (pre-consent). After the visitor clicks Reject All, the site goes on to fire 185 requests in total. The delta is 99 - well above the absolute floor of 10 and the relative floor of 10% (99 / 86 = 115%). Both gates clear, so a leak Finding is recorded.
Sites that clear only one gate do not trigger a Finding. The two-gate design is deliberate - it stops single-pixel reload noise on small sites from registering, and it stops heavy-content sites from registering simply because 10-20 background requests is normal post-banner behaviour.
Severity ladder
When a Finding is recorded, the size of the leak determines the maximum grade the site can receive. The ladder is one-way - a Finding can only cap a grade, never raise it.
| Observation | Severity | Grade cap | Reading |
|---|---|---|---|
| Δ ≤ 25 extra requests | Moderate | C | Reject is not clean, but the leak is contained. |
| Δ ≤ 50 extra requests | High | D | Reject produces only token suppression of third-party traffic. |
| Δ > 50 extra requests | Critical | F | Reject is effectively inert. Most third-party traffic continues unchanged. |
| Reject button cannot be operated | High | D | A consent platform is detected but no functional reject control could be exercised. |
When more than one Finding fires on the same scan, the strictest cap wins. A critical leak combined with a non-functional reject button caps at F.
What this is, and what it isn't
What the grade is
A reproducible, evidence-backed editorial signal. A summary of observable browser behaviour - which trackers fire, when they fire, and whether reject is honoured. Each grade is derived from a versioned set of Findings that the scan emits.
What the grade is not
A regulatory judgement. ConsentMark does not declare a site lawful or unlawful. The Findings are observations of measurable browser behaviour; any determination about lawfulness sits with the regulator, the controller, and their advisors. The evidence chain is the legal artefact - the letter is the headline.
Why these signals matter
The reject-state leak threshold is grounded in published regulator guidance and enforcement decisions, not in any private interpretation:
- EDPB Cookie Banner Taskforce - Report of work undertaken (Report 2/2023)
European Data Protection Board guidance on the practical application of Article 5(3) of the ePrivacy Directive and the Article 7 GDPR consent obligations to cookie banners and trackers.
- CNIL sanctions Google and Facebook over cookie practices (2021)
French data protection authority fines for cookie banners that did not provide a reject option as easy as the accept option, and for tags firing before consent.
What is not in v2 yet
v2 is deliberately narrow. Three signals are visible in the report as evidence but do not yet move the grade:
- Pre-consent surface penalty. The discovery-phase tag inventory is recorded but not yet weighted as its own scoring dimension. Planned for v1.1.
- Cookie-delta scoring. Cookie counts before and after reject are carried as supporting evidence on every leak Finding. Cookie behaviour is noisier than request behaviour, so it stays as context until the signal is well enough characterised to score on.
- Strictly-necessary vendor classification. Until each vendor can be classified as strictly-necessary or not, the leak threshold counts all post-reject requests rather than only non-strictly-necessary ones.
Versioning and audit trail
Every scan result carries a scoringEngineVersion field (currently v1 or v2). When the engine changes, older /scan/:id URLs keep their original grade so the audit trail stays intact - a grade attached to a specific scan ID is the grade as it was recorded on that date, under the engine version in force at the time. Re-running a scan against the latest engine produces a new scan ID with its own version stamp.
Disputing a grade
If you believe a published scan misrepresents your site, write to contact@consentmark.com with the scan URL and the specific Finding you contest. We commit to:
- Acknowledging the dispute within two working days.
- A 14-day notification window before any change to the public grade of an affected site, where a regrade is the result of a methodology change rather than a fresh scan.
- Preserving the original scan record alongside any correction, so the audit trail is never silently rewritten.