About this project
Ditto presents itself as "Semantic CI": an extra pipeline question alongside compile, test and lint — is this codebase reinventing something it already knows? It targets Type-4 clones, functions that behave the same but are written differently, which token- and AST-based duplicate detectors generally do not report.
How the documented pipeline works: the backend downloads a repository tarball, walks it with ts-morph to extract every function including non-exported ones, then fingerprints each function one at a time with a cheap model in a name-blind way. Those fingerprints — never raw code or names — are embedded and clustered by cosine similarity in memory. Only the resulting candidate clusters reach a larger model for adjudication, which also proposes adversarial inputs. Functionally pure functions are then executed side by side in a worker_threads sandbox with a timeout, and divergence confirmed there is presented as executed evidence; impure functions are still clustered and adjudicated but labelled as predicted, not executed. Results are written to MongoDB and served by read-only endpoints plus a Next.js frontend; the serving path calls no model.
The README reports runs across five repositories (2,870, 2,654, 336, 31 and 6 functions), listing duplicate clusters, behavioural conflicts and 18 cases proven by execution, including a family of truncateText implementations in cline where a reserved-space calculation collapses kept text to one character above a certain limit. It states that jscpd reports zero clones across those files. Two small, well-maintained libraries scored as clean, which the authors present as evidence against over-reporting.
Stated limits: JavaScript/TypeScript only, since the AST layer is ts-morph; execution requires purity; large repositories must be scoped explicitly rather than truncated, because a silently dropped function can make a whole cluster disappear; and the tool does not recommend which of two conflicting implementations to keep, framing that as a human decision. Cost and timing figures in the README (for example ₹232 for a 2,870-function analysis, and roughly ₹1 per pull request for the planned Guard check) are the project's own measurements, not independent benchmarks. The hosted demo caps on-demand analysis at 600 functions, described as a limit on the maintainers' API credits; running locally with your own key removes the cap.
Setup notes cover indexing a repo without MongoDB or an API key, an .env with MongoDB, OpenAI and optional GitHub/model settings, Cloud Run deployment with a 1200-second request timeout, Atlas network access, and Vercel for the frontend where NEXT_PUBLIC_* values are inlined at build time. A roadmap lists Ditto Guard as a GitHub Action pull-request check, an MCP tool letting coding agents query the index before writing duplicates, additional languages via tree-sitter, and incremental re-indexing. Contributors are pointed to labelled good first issues.
Comments
0 Rating appears after 10 ratings
Sign in to join the discussion.