Measuring facial retouching and synthetic faces: the TrueFace signal pipeline

1. Scope and positioning

TrueFace produces detection signals about a portrait: likelihood of beauty filters or retouching, likelihood of AI generation or face swap, and which facial regions appear reshaped. It does not verify identity, does not perform KYC, and does not issue legally binding conclusions. In NIST SP 800-63A-4 terms, its output is supporting evidence a human or a downstream proofing system may weigh — not an identity-proofing decision.

2. Pipeline overview

1. Face localisation and classification — one vision-model inference per scan returns, for each face, a bounding box, a filter probability, an AI/deepfake assessment and a per-region reshaping rating (jaw, nose, eyes, brows).
2. Deterministic skin-detail measurement — computed in the user's browser, no AI and no network cost (section 3).
3. Decision rules — the probability is mapped to a verdict band, then downgraded to Uncertain whenever independent evidence disagrees (section 4).
4. Optional bounded reconstruction for paid users — exactly one edit call and one fail-closed review call (section 6).

3. Skin micro-texture ratio

Beauty filters suppress high-frequency skin detail while usually leaving hair, clothing and background sharp. We therefore compare fine detail inside the face with a ring around it in the same photo, which cancels most camera, compression and focus differences.

• The image is rescaled so the face is about 370 px wide; faces narrower than 160 px in the source are not measured.
• Luma L = 0.299R + 0.587G + 0.114B; a separable Gaussian blur (σ = 1.5 px) gives G(L).
• Detail per pixel is |L − G(L)|. The inner face is the ellipse at normalised distance ≤ 0.6; the reference ring spans 1.3–2.2.
• Ratio R = mean(inner) / mean(ring). Below 0.42 the face is treated as smoother than its surroundings.

This signal is deliberately one-directional: a low ratio can only move a confident “natural” verdict to Uncertain. It never asserts a filter on its own, because strong compression or noise reduction also lowers it on natural photos. The 0.42 cutoff was set on a very small number of reference pairs and is provisional.

4. Verdict bands and uncertainty rules

A verdict is replaced by Uncertain, and the triggering rule is shown to the user and stored in an automatic 30-day report, when:

• lighting conditions fall in a band known to distort texture cues (e.g. strong direct daylight);
• independent checks of the same face disagree — we show the disagreement rather than pick the most plausible run;
• the skin-detail ratio says the skin is smoother than its surroundings while the model says natural.

5. Decision boundary and calibration audit

We weight a false accusation of manipulation as roughly five times more costly than a missed filter (C_FP ≈ 5 × C_FN), because wrongly accusing a genuine person causes more harm in trust workflows. The live cutoffs (0.52 and 0.18) are currently hand-set; they have not yet been re-optimised against this cost ratio, because the labelled sample is too small to do so reliably.

Preliminary labelled sample (N = 70 conclusive labels)

Interpretation: a “filters” flag is usually right (PPV ≈ 95%), but a “natural” result is weak evidence (NPV ≈ 65%). The high ECE shows raw probabilities are not yet well calibrated (Guo et al., 2017), and scores cluster at a few values. Both false alarms were natural photos in bright daylight. The sample comes from a single labeller and may contain related photos; these figures are a baseline, not a performance claim, and will be recomputed as users contribute verified labels.

6. Bounded forensic reconstruction

Reconstruction estimates how a face may look without retouching. It is an estimate, never ground truth.

• Evidence-authorised edits only — geometric reversal (jaw, nose, eyes, brows) is attempted only for regions the scan flagged; texture and tone changes stay minimal when evidence is weak.
• Fixed compute budget — exactly one edit call plus one review call. No retries.
• Fail-closed review — output is rejected unless the reviewer rates it the same person at ≥ 70% and not beautified.
• Pixel texture gate — output is rejected if inner-face fine detail falls below 80% of the source (MIN_TEXTURE_RETAINED = 0.8), because an AI reviewer cannot reliably see skin smoothing.

7. Standards context

ISO/IEC 30107-3 defines how presentation attack detection is tested and reported. TrueFace has notbeen tested or certified against it; we use its vocabulary (attack presentation, bona fide presentation, error rates reported separately) to describe our signals honestly. Results are reported per face so that one filtered person in a group photo does not mislabel the others.

8. Privacy and security boundary

• The skin-detail measurement runs entirely in the browser.
• Uploaded images are removed from the processing cache after analysis; history thumbnails are kept only when the user enables history, for up to 30 days.
• API keys and model gateway credentials exist only server-side; database access is restricted per user by row-level policies.

See the privacy policy for full retention terms.

9. Known limitations

• Small, single-labeller calibration set; cutoffs are provisional.
• Heavy recompression, screenshots of screenshots and low resolution reduce every signal.
• Bright direct light can make natural skin read as smoothed.
• Reconstruction cannot recover detail the filter destroyed; it estimates it.