Token-Based Clone Detection for PHP: An Empirical Evaluation of Methods and a Labelled Benchmark

phpcpd is a widely used PHP copy/paste (clone) detector whose last released version (6.0.3, 2020) performs Rabin–Karp exact matching of Type-1 and Type-2 clones. An unreleased 7.0.0 development line added a ConQAT-style approximate suffix-tree algorithm for Type-3 (near-miss) detection (contributed upstream as pull request #199), but 7.0.0 was never released and the project was archived in 2023, so this capability never reached users. The clone-detection literature offers further methods — order-invariant token-bag matching and type-aware normalization — that had not been applied to PHP, and, unlike Java (BigCloneBench) or C (POJ-104), no labelled PHP clone benchmark existed on which to evaluate any of them. This work addresses both gaps. Building on the archived codebase, we modernize it to PHP 8.5, and complete and evaluate its previously unreleased suffix-tree engine; we implement additional literature-grounded methods as selectable engines (a SourcererCC token-bag matcher, type-aware edit weights, a token-type abstraction with a type-anchored variant, an IDF stopword filter, and an incremental index that provably reproduces a full pass); and we construct BCB-PHP, a labelled PHP clone benchmark (to our knowledge the first), comprising a mutation-injection harness over a pinned, six-corpus type-density gradient. Evaluating each method on BCB-PHP yields a type-density × clone-density map and a specificity increase of +10.9 to +45.3 points from type-anchored normalization at no recall cost, motivating a combined exact-and-reorder-tolerant default configuration; identifier fuzzing is shown to reduce precision substantially. A profiler-guided banded edit-distance optimization further cuts suffix-tree matching cost by ~3.5× with byte-identical output. The tool is released under BSD-3-Clause, and BCB-PHP is released for reuse.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-06-28
DOI
https://doi.org/10.5281/zenodo.20980135
Primary Topic
Software Engineering Research
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Token-Based Clone Detection for PHP: An Empirical Evaluation of Methods and a Labelled Benchmark

Luciano Pereira
Zenodo (CERN European Organization for Nuclear Research)
Software Engineering Research
preprint

Token-Based Clone Detection for PHP: An Empirical Evaluation of Methods and a Labelled Benchmark

Luciano Pereira
preprint en

Abstract

phpcpd is a widely used PHP copy/paste (clone) detector whose last released version (6.0.3, 2020) performs Rabin–Karp exact matching of Type-1 and Type-2 clones. An unreleased 7.0.0 development line added a ConQAT-style approximate suffix-tree algorithm for Type-3 (near-miss) detection (contributed upstream as pull request #199), but 7.0.0 was never released and the project was archived in 2023, so this capability never reached users. The clone-detection literature offers further methods — order-invariant token-bag matching and type-aware normalization — that had not been applied to PHP, and, unlike Java (BigCloneBench) or C (POJ-104), no labelled PHP clone benchmark existed on which to evaluate any of them. This work addresses both gaps. Building on the archived codebase, we modernize it to PHP 8.5, and complete and evaluate its previously unreleased suffix-tree engine; we implement additional literature-grounded methods as selectable engines (a SourcererCC token-bag matcher, type-aware edit weights, a token-type abstraction with a type-anchored variant, an IDF stopword filter, and an incremental index that provably reproduces a full pass); and we construct BCB-PHP, a labelled PHP clone benchmark (to our knowledge the first), comprising a mutation-injection harness over a pinned, six-corpus type-density gradient. Evaluating each method on BCB-PHP yields a type-density × clone-density map and a specificity increase of +10.9 to +45.3 points from type-anchored normalization at no recall cost, motivating a combined exact-and-reorder-tolerant default configuration; identifier fuzzing is shown to reduce precision substantially. A profiler-guided banded edit-distance optimization further cuts suffix-tree matching cost by ~3.5× with byte-identical output. The tool is released under BSD-3-Clause, and BCB-PHP is released for reuse.

Zenodo (CERN European Organization for Nuclear Research)
Software Engineering Research
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.