top companies

{
  "id": "P15-C-001",
  "name": "Azure AI Personalizer",
  "category": "bandits",
  "source_url": "https://learn.microsoft.com/en-us/azure/ai-services/personalizer/what-is-personalizer",
  "observed_date": "2026-08-27",
  "evidence_class": "first_party_docs",
  "claim": "Managed contextual-bandit service exposing a two-call Rank/Reward contract; retiring 25 August 2026 with migration recommended to open-source microsoft/learning-loop.",
  "feedback_signals": "Reward API: a real-valued score 0-1 per event ID, sent immediately or delayed; context features and action features supplied on each Rank call.",
  "what_it_updates": "An underlying reinforcement-learning policy per 'Learning Loop' resource, updated in near-real time to maximize average reward.",
  "loop_evidence": "documented_methodology",
  "cold_start_approach": "Apprentice mode: the model observes the application's existing baseline decision logic and learns to mimic it before taking online control, explicitly framed as cold-start mitigation. Also states a practical floor of ~1,000 events/day to learn effectively.",
  "privacy_approach": "Learns from collective information across all users; docs explicitly state it does not persist user profiles, does not log individual preferences/history, and that unique user IDs should not be sent.",
  "limitations": "Retired as of 25 Aug 2026 (two days before observation date) and closed to new resources since 20 Sep 2023; capped at ~50 actions per Rank call, so it requires an upstream retrieval stage for larger candidate sets. Note: the Azure product marketing page and some Learn pages carry a conflicting 1 October 2026 date — the current what-is-personalizer page states 25 August 2026.",
  "disposition": "top10",
  "top10_rank": 2,
  "top10_rationale": "The closest published Rank/Reward contract to P15's core loop, and its Apprentice mode remains the best cold-start answer in the census: rank using the existing metadata heuristic while silently scoring the bandit against it, handing over control only once it beats the baseline. Now ranked second rather than first because it is retired as of 25 August 2026 — a live mechanism (Renovate) that learns at the asset-version level outranks a decommissioned one that learned per-decision. Its retirement is itself load-bearing evidence: Microsoft closed new resources in Sep 2023 and shut the service down, directing users to the open-source microsoft/learning-loop rather than a managed successor, which says the managed contextual-bandit category did not sustain. Its stated ~1,000 events/day floor and ~50-action cap remain the honest calibration data arguing against per-asset bandits at Actionist's scale.",
  "sparse_data_fit": "Splits cleanly. The bandit itself FAILS: Microsoft's own page recommends '~1,000 events per day to enable Personalizer to learn effectively', roughly four orders of magnitude beyond a tens-of-builds regime, and caps Rank calls at ~50 actions so a larger shelf needs an upstream retrieval stage anyway. Apprentice mode SURVIVES and is the single best cold-start pattern in the census: the system returns the application's existing baseline decision while silently scoring itself against it, taking control only once it demonstrably beats that baseline. That requires no minimum data volume to be safe — it is safe by construction, because until the learner is proven the user gets the heuristic. Actionist can adopt it immediately: rank the shelf by metadata heuristic, log what the evidence layer would have chosen, and switch only when the shadow ranking wins."
}

{
  "id": "P15-C-002",
  "name": "Algolia Dynamic Re-Ranking",
  "category": "ranking_service",
  "observed_date": "2026-08-27",
  "source_url": "https://www.algolia.com/doc/guides/algolia-ai/re-ranking",
  "evidence_class": "first_party_docs",
  "claim": "Reranks an already-eligible result set using click and conversion events over a sliding window, with published minimum-evidence thresholds before any record is promoted.",
  "feedback_signals": "Click and conversion events that carry a queryID tying them back to a specific Algolia request; events without a queryID are invalid for the loop. Optional 'event freshness' weighting favours recent events.",
  "what_it_updates": "Per-query ordering of the top N results (default first 20), refreshed every 24 hours by default. It only reorders records already eligible from textual relevance, filters, and custom ranking — it never injects new records.",
  "loop_evidence": "documented_methodology",
  "cold_start_approach": "Explicit evidence floor rather than a prior: a record needs at least 20 clicks or 2 conversions for a given query within a 30-day window before it is promoted; below that, the deterministic base ranking simply stands. Algolia documents that this typically needs ~20K events/30 days overall.",
  "privacy_approach": "unknown",
  "limitations": "Per-query loop, so evidence does not pool across related queries — sparse or long-tail queries never accumulate enough events to rerank, which Algolia surfaces as a 'Low Re-Ranking Effect' warning.",
  "disposition": "top10",
  "top10_rank": 10,
  "top10_rationale": "Retains a top-10 slot for one architectural idea that is exactly Actionist's problem shape: a deterministic base ranking produces the candidate set, and a production-evidence layer reorders only what already qualified, never injecting a record the deterministic rules rejected. That two-stage split is the safest way to add outcome ranking to a metadata-only shelf. Ranked tenth rather than second after re-ranking against the full set because its published evidence floors — 20 clicks or 2 conversions per query in a 30-day window, ~20K events/30 days overall — put it firmly in the volume-dependent group; alongside GrowthBook it is confirmation that click-driven reranking does not reach a tens-of-builds regime. The transferable residue is the containment property and the willingness to publish a refusal-to-rerank threshold, not the thresholds themselves.",
  "sparse_data_fit": "FAILS on its published thresholds, which is precisely why the record is useful. A record needs at least 20 clicks or 2 conversions for a given query within a 30-day window before promotion, and Algolia states the feature typically needs ~20K events per 30 days overall. Worse for Actionist, the loop is per-query so evidence never pools across related queries — long-tail queries never accumulate enough, surfaced by Algolia's own 'Low Re-Ranking Effect' warning. That is the shelf-asset problem exactly: most assets appear in few builds. What survives independent of volume is the two-stage containment architecture — the evidence layer only reorders records the deterministic layer already admitted, and can never inject one it rejected — plus the discipline of publishing an explicit refusal-to-rerank threshold rather than silently scoring on thin data."
}

{
  "id": "P15-C-003",
  "name": "Statsig Autotune",
  "category": "bandits",
  "source_url": "https://docs.statsig.com/autotune/multi-armed-bandit",
  "observed_date": "2026-08-27",
  "evidence_class": "first_party_docs",
  "claim": "Multi-armed bandit that allocates traffic across variants in proportion to each variant's posterior probability of being best, using Thompson sampling.",
  "feedback_signals": "A single target/goal metric per Autotune experiment (e.g. a button-click conversion event); for the contextual variant, all custom attributes on the Statsig user object are fed into model training.",
  "what_it_updates": "Traffic allocation across variants, continuously; the process terminates when the winning variant beats the runner-up by a specified margin.",
  "loop_evidence": "documented_methodology",
  "cold_start_approach": "CORRECTED 2026-08-27 on direct fetch of the docs page: no exploration window, warm-up period or default duration is documented. The stopping condition is margin-based, not time-based — the process concludes 'When the winning treatment beats the second-best treatment by a specified margin', and that margin's default is not stated. The previously recorded '24 hour default exploration window' was not supported by the page and has been removed.",
  "privacy_approach": "unknown",
  "limitations": "Statsig documents the tradeoff candidly: the non-contextual bandit converges to a single global winner and cannot personalize, so where user attributes interact with variants it can settle on a global maximum worse than the per-user best. CORRECTION: the earlier version of this record cited a 24-hour default exploration window and flagged the Thompson-sampling detail as unverified search-summary evidence. The docs page has now been fetched directly — Thompson sampling and proportional-to-probability-of-best allocation are confirmed first-party ('uses a Thompson Sampling (Bayesian) algorithm to estimate each variant's probability of being the best variant', with a worked example where a 60%-probability variant receives 60% of traffic) — but no exploration window is documented and that claim is withdrawn.",
  "disposition": "census",
  "top10_rank": null,
  "top10_rationale": null,
  "sparse_data_fit": "FAILS per Statsig's own published floors, though it is the least demanding mechanism in the bandits tier. The docs' comparison table states the multi-armed bandit needs 'Very Little (100+ samples)', the contextual bandit 'Little - generally 1000+ samples', and the ranking engine substantially more. 100 samples per arm is still an order of magnitude beyond a shelf asset appearing in a handful of builds. What does transfer without volume is the Thompson-sampling posture itself: with little evidence the posteriors stay wide and allocation stays near-uniform, so a new asset is explored rather than buried, and no epsilon needs hand-tuning. That graceful degradation is the property worth copying even if the statistical machinery is not run."
}

{
  "id": "P15-C-004",
  "name": "Eppo Contextual Bandits",
  "category": "bandits",
  "source_url": "https://docs.geteppo.com/contextual-bandits/",
  "observed_date": "2026-08-27",
  "evidence_class": "first_party_docs",
  "claim": "Contextual bandit that predicts the objective metric per action from subject/action context, then selects with injected randomness, retraining on a scheduled cadence.",
  "feedback_signals": "An operator-chosen optimization metric measured over a lookback window, joined to logged assignment contexts; Eppo advises choosing a short-horizon metric correlated with the long-term outcome because bandits need fast feedback.",
  "what_it_updates": "Bandit model parameters on a batch schedule (documented example: retrain every 24 hours over a 30-day lookback), which then changes serving frequency per action.",
  "loop_evidence": "documented_methodology",
  "cold_start_approach": "unknown",
  "privacy_approach": "unknown",
  "limitations": "Batch retraining on a ~24h cadence means the policy is stale between runs; Eppo also documents that because the bandit personalizes, action-vs-action comparisons are not meaningful, so evaluation must be bandit-vs-holdout-control rather than per-arm.",
  "disposition": "census",
  "top10_rank": null,
  "top10_rationale": null,
  "sparse_data_fit": "FAILS as a statistical mechanism; Eppo advises choosing a short-horizon metric precisely because 'bandits need fast feedback', and a documented example retrains every 24 hours over a 30-day lookback — a cadence assuming continuous traffic Actionist does not have. Two operational ideas survive at any scale. First, the explicit separation of real-time selection from periodic batch model updating: for Actionist, reranking the shelf nightly is entirely sufficient and a streaming policy would be over-engineering. Second, and more important, Eppo's insistence that because the bandit personalizes, action-vs-action comparisons are not meaningful, so evaluation must be bandit-vs-holdout-control. Keeping some builds on metadata-only ranking to prove the evidence layer actually helps is cheap, works at tens of builds, and is the only way to know the loop is not making things worse."
}

{
  "id": "P15-C-005",
  "name": "Shopify App Store search ranking",
  "category": "marketplace_ranking",
  "source_url": "https://shopify.dev/docs/apps/launch/marketing/manage-app-reviews",
  "observed_date": "2026-08-27",
  "evidence_class": "first_party_docs",
  "claim": "Marketplace ranking for reusable software components where merchant-side outcome signals, not just metadata, determine placement; Shopify first-party docs confirm a recency- and trust-weighted rating feeds search placement.",
  "feedback_signals": "First-party confirmed: merchant reviews, weighted to prioritize recent, useful, and trustworthy reviews rather than a flat star average. Widely reported but NOT first-party confirmed: install volume/velocity, listing conversion rate, uninstall/retention rate, post-install active usage.",
  "what_it_updates": "Position in App Store search results and category pages; Shopify also documents demotion as an enforcement lever against review manipulation.",
  "loop_evidence": "documented_methodology",
  "cold_start_approach": "unknown",
  "privacy_approach": "unknown",
  "limitations": "Only the review-weighting half is first-party documented. The behavioral factors (installs, retention, CTR) that make this interesting for P15 come from agency SEO blogs and third-party observation, not Shopify — they should be treated as secondary and unverified. The full algorithm and its weights are not published.",
  "disposition": "census",
  "top10_rank": null,
  "top10_rationale": null,
  "sparse_data_fit": "MIXED, and the first-party-confirmed half is the half that survives. Shopify documents weighting reviews to prioritise recent, useful and trustworthy ones rather than taking a flat star average — recency-and-trust weighting needs no volume and directly addresses Actionist's staleness problem, where an asset that worked well across twenty builds two years ago should not outrank one succeeding now. The behavioural signals that would make this a true production loop (install velocity, listing conversion, uninstall and retention rates) are third-party observation, not Shopify documentation, and in any case are volume-dependent in a way a shelf of tens of assets cannot support. Treat the weighting scheme as adoptable and the behavioural ranking as unverified."
}

{
  "id": "P15-C-006",
  "name": "Mend Renovate Merge Confidence",
  "category": "registry_health",
  "source_url": "https://docs.renovatebot.com/merge-confidence/",
  "observed_date": "2026-08-27",
  "evidence_class": "first_party_docs",
  "claim": "Attaches Age/Adoption/Passing/Confidence badges to dependency-update PRs, computed from aggregated real-world CI outcomes across the hosted Renovate fleet, in order to 'find and flag undeclared breaking releases'.",
  "feedback_signals": "Production outcomes of the same upgrade attempted elsewhere: whether the update's tests passed in other users' repositories ('Passing' = the percentage of updates which have passing tests for this package), what share of Renovate users have adopted the release ('Adoption'), and package age. Derived from the hosted app's PR corpus (docs cite millions of PRs created on github.com since 2017).",
  "what_it_updates": "A per-package-per-version confidence level (Low / Neutral / High / Very High) surfaced as a PR badge, and — for paying customers via Merge Confidence Workflows — automated gating rules such as only raising a PR once an update reaches High confidence, or auto-merging Very High.",
  "loop_evidence": "documented_methodology",
  "cold_start_approach": "Explicit 'Neutral' state meaning 'We don't have enough data about the update', rather than a fabricated score. Additionally a hard time gate: npm packages cannot receive the High badge until at least three days old, because packages under three days can be unpublished.",
  "privacy_approach": "Aggregation across the fleet; percentages are not raw counts but 'weighted towards Organizations, private repositories, and projects with high test reliability'. Individual repository outcomes are not exposed — only the pooled percentage.",
  "sparse_data_fit": "STRONG on mechanism, weak on Mend's own data scale. The transferable parts need no volume at all: an explicit Neutral/insufficient-evidence state, a minimum-age gate before an asset can be promoted, and weighting evidence by source reliability rather than counting it flat. Actionist can implement all three at ten builds. The Adoption/Passing percentages themselves are the part that needs a fleet, and Actionist will not have one for years.",
  "limitations": "The scoring function is deliberately private — docs state 'The algorithm that decides on the values is private and is not something we plan to share', comparing it to a search engine ranking. So the badge inputs are documented but the weights and model form are not reproducible. Confidence is also only available for a fixed datasource list (go, npm, maven, pypi, nuget, packagist, rubygems), and the richer Workflows gating is paywalled.",
  "disposition": "top10",
  "top10_rank": 1,
  "top10_rationale": "The single closest commercial precedent to P15's core ask: production outcomes of using a software asset are pooled across many independent projects and fed back as a recommendation about whether to adopt that asset. That is precisely 'accepted plans, incidents and maintenance outcomes rerank the shelf', already shipped. Three design choices transfer to Actionist immediately and survive sparsity: a first-class Neutral state that refuses to score on thin evidence rather than guessing, a minimum-age gate that blocks promotion regardless of how good early signals look, and weighting evidence by the reliability of its source instead of counting observations flat. Ranked above Personalizer because it is live rather than retired, and because its unit of learning is the asset-version — exactly Actionist's unit — rather than a per-user personalization decision."
}

{
  "id": "P15-C-007",
  "name": "Socket.dev package scores",
  "category": "registry_health",
  "source_url": "https://docs.socket.dev/docs/package-scores",
  "observed_date": "2026-08-27",
  "evidence_class": "first_party_docs",
  "claim": "Publishes the actual scoring formula, including per-severity normalization functions, soft caps, and a power-scaling exponent gamma derived from project size and popularity.",
  "feedback_signals": "Not production outcomes. Static analysis alerts from an in-house engine (install scripts, network access, obfuscated code, typosquat signals) plus registry/repo metadata. The only adoption-like signals are Download Count (weight 5 under Supply Chain Risk, normalization -(e^(-x/10000)-1)) and Stargazers/Forks/Watchers (weight 1 each).",
  "what_it_updates": "Five per-version category scores out of 100 — Supply Chain Risk, Quality, Maintenance, Vulnerabilities, License. Scores are versioned per package release.",
  "loop_evidence": "documented_methodology",
  "cold_start_approach": "No separate cold-start path; a new package is simply scored on whatever static signals exist. The gamma term means an unpopular package is penalised harder for the same alert than a popular one.",
  "privacy_approach": "not_applicable — operates on public registry artifacts, no user data in the loop.",
  "sparse_data_fit": "The gamma mechanism is the transferable idea and it is explicitly sparsity-aware: gamma ~= 1/2 + c0*log(lines of code) + c1*log(popularity), and 'Larger or more widely used packages tend to have a smaller gamma', which 'reduces the impact of negative metrics'. That is confidence-weighting by evidence volume expressed as an exponent — a well-used asset is not condemned by one bad signal, a barely-used one is. Directly applicable to a shelf where some assets have twenty builds behind them and others have one. But note this is NOT a production-outcome loop; it is static scoring with a popularity-aware confidence dial.",
  "limitations": "No production or downstream outcome data whatsoever — an asset that repeatedly breaks in real deployments scores identically to one that never does, provided the code looks the same. Coefficients c0 and c1 are not published, only the formula shape. Docs carry an explicit staleness caveat that the page 'may not exactly represent the scoring system as deployed in Socket at this point in time'.",
  "disposition": "top10",
  "top10_rank": 6,
  "top10_rationale": "Included for one specific, unusually well-published mechanism rather than for its loop, which does not exist: Socket is the only vendor in this census that prints the algebra of how confidence in an asset scales with how much evidence backs it. The gamma exponent answers a question Actionist will hit immediately — how do you avoid letting a single bad build bury an asset that has succeeded twenty times, while still reacting fast to an asset with one data point? Ranked sixth rather than higher because it is honestly a static-analysis scorer: nothing about production outcomes enters it, so it informs the weighting layer of P15 (a) and nothing else."
}

{
  "id": "P15-C-008",
  "name": "ecosyste.ms",
  "category": "registry_health",
  "source_url": "https://packages.ecosyste.ms/api/v1/registries/npmjs.org/packages/react",
  "observed_date": "2026-08-27",
  "evidence_class": "observed_behavior",
  "claim": "Exposes per-package percentile rankings across real adoption signals via public API, alongside raw dependent and download counts and a boolean criticality flag.",
  "feedback_signals": "Downstream adoption rather than outcomes: dependent_packages_count, dependent_repos_count, downloads (last-month period), docker_dependents_count, docker_downloads_count, plus repo and issue metadata (avg_time_to_close_issue, avg_time_to_close_pull_request).",
  "what_it_updates": "A 'rankings' object holding percentile ranks per signal — observed on the react record as downloads 0.0161, dependent_repos_count 0.0070, dependent_packages_count 0.000055, stargazers_count 0.0039, forks_count 0.0028 — i.e. the package's position in the whole-registry distribution per axis. Also a 'critical' boolean (true for react).",
  "loop_evidence": "documented_methodology",
  "cold_start_approach": "unknown — no documented prior or floor; a package with no dependents simply ranks at the bottom of the distribution.",
  "privacy_approach": "Public data only; data published CC BY-SA 4.0, code AGPL-3.",
  "sparse_data_fit": "The transferable mechanism is percentile-rank-within-corpus rather than absolute score. This is genuinely sparse-friendly: with 30 shelf assets you can still say 'this asset is in the top decile of the shelf for successful reuse', which is a meaningful and stable statement, whereas an absolute score of 0.62 is not. Percentile ranking also degrades honestly — it never claims more precision than the corpus supports. The caution is that with tens of assets, percentile buckets are coarse and a single new observation can move an asset several places, so ranks need smoothing or banding.",
  "limitations": "Adoption is not outcome — dependent_repos_count says many projects installed it, never that it worked. No incident, failure, or removal signal at all. The 'critical' flag's derivation was not verified from first-party docs in this run; only its presence in the API response was observed. Values quoted are a single observed record (react, last_synced_at 2026-08-26), not a documented schema guarantee.",
  "disposition": "census",
  "top10_rank": null,
  "top10_rationale": null
}

{
  "id": "P15-C-009",
  "name": "npms.io scoring (npms-analyzer)",
  "category": "registry_health",
  "source_url": "https://github.com/npms-io/npms-analyzer/blob/master/docs/architecture.md",
  "observed_date": "2026-08-27",
  "evidence_class": "first_party_docs",
  "claim": "Published an unusually explicit normalization method: each package's raw evaluation is normalized against the corpus min/max, then mapped through a Bezier curve anchored on the corpus mean.",
  "feedback_signals": "Three scored categories. Popularity: stars, forks, subscribers, contributors, dependents, downloads, and 'downloads acceleration'. Quality: README/license presence, version stability, test coverage, build status, outdated or vulnerable dependencies, badges, linters. Maintenance: ratio of open to total issues, issue close times, most recent commit, commit frequency.",
  "what_it_updates": "A per-package composite score used to order npm search results, recomputed by a 'continuous scoring' process separate from per-package evaluation.",
  "loop_evidence": "documented_methodology",
  "cold_start_approach": "Explicitly deferred rather than faked: a package analyzed before any corpus aggregation exists 'won't be scored at the moment' and is picked up on a later scoring pass. No score is invented for an unranked package.",
  "privacy_approach": "not_applicable — public registry data only.",
  "sparse_data_fit": "STRONG, and the most directly stealable formula in the registry tier. The aggregation step computes min, max and mean across the whole corpus, normalizes each package into [0,1] against those, then scores via a Bezier curve with control points (0,0), (mean,0.75), (mean,0.75), (1,1) — so a package sitting exactly at the corpus mean lands near 0.75. This is relative-to-cohort scoring, which is exactly what works at tens of observations: you never need an absolute threshold, only the shelf's own distribution. The doubled middle control point also compresses the middle of the distribution, meaning small evidence differences near the mean do not produce large rank swings — a useful anti-jitter property when one new build arrives.",
  "limitations": "The project is effectively unmaintained and npms.io's public service was retired; this is a historical methodology document, not a live product. The doc does not publish the numeric weights blending quality/popularity/maintenance into the composite, so the blend is unreproducible. A fourth 'personalities' category is described but was never built. Popularity signals are adoption, not outcome.",
  "disposition": "top10",
  "top10_rank": 5,
  "top10_rationale": "Contributes the cleanest published answer to the hardest sparse-data question in P15: how do you turn a handful of raw observations into a score that is stable and comparable? Its answer — normalize against the corpus's own min/max/mean and shape the result with a curve that flattens near the mean — needs no minimum event count, only a corpus, and Actionist's shelf is a corpus from day one. The refusal to score a package before aggregation exists is the same discipline as Renovate's Neutral state, arrived at independently, which is a strong signal that an explicit unscored state is the right default rather than a nicety. Ranked fifth because it is dead software and the composite weights were never published, so only the normalization idea survives."
}

{
  "id": "P15-C-010",
  "name": "OpenSSF Scorecard",
  "category": "registry_health",
  "source_url": "https://github.com/ossf/scorecard",
  "observed_date": "2026-08-27",
  "evidence_class": "first_party_docs",
  "claim": "Scores repositories 0-10 per check and aggregates by risk weight, entirely from static repository and project metadata — no downstream or production usage signal enters the score.",
  "feedback_signals": "None from production. Checks inspect repo contents and configuration: Binary-Artifacts, Branch-Protection, CI-Tests, Code-Review, Dangerous-Workflow, Dependency-Update-Tool, Fuzzing, License, Maintained, Pinned-Dependencies, Packaging, SAST, Security-Policy, Signed-Releases, Token-Permissions, Vulnerabilities (via OSV), Webhooks. Maintained and Contributors use project-internal activity, not consumer outcomes.",
  "what_it_updates": "An aggregate score described as 'a weight-based average of the individual checks weighted by risk', with weights Critical 10, High 7.5, Medium 5, Low 2.5.",
  "loop_evidence": "documented_methodology",
  "cold_start_approach": "not_applicable — every repo is scorable immediately because all inputs are static; there is no evidence-accumulation phase.",
  "privacy_approach": "not_applicable — public repositories only.",
  "sparse_data_fit": "Trivially fits sparse regimes because it never learns: the score is a pure function of current repo state, so it works identically at one observation or ten thousand. That is exactly why it is a NEGATIVE result for P15 — it is the baseline that a production-learning loop must beat. Its real contribution to Actionist is the risk-weighted aggregation shape (bucket checks by severity tier, average weighted by tier) which is a defensible way to combine heterogeneous static metadata signals about a shelf asset before any outcome evidence exists.",
  "limitations": "Explicitly not an outcome loop. The project itself warns the aggregate 'tells you nothing about what individual behaviors a repository is or is not doing' and recommends probe-level results over the composite; checks are acknowledged heuristics with false positives and negatives. Weekly bulk scans omit CI-Tests, Contributors and Dependency-Update-Tool for API cost, so published API scores do not reflect the full default check set.",
  "disposition": "census",
  "top10_rank": null,
  "top10_rationale": null
}

{
  "id": "P15-C-011",
  "name": "deps.dev (Google Open Source Insights)",
  "category": "registry_health",
  "source_url": "https://docs.deps.dev/faq/",
  "observed_date": "2026-08-27",
  "evidence_class": "first_party_docs",
  "claim": "Builds annotated dependency graphs from first principles without building or installing software; provides no scoring system and no production feedback loop.",
  "feedback_signals": "None. Data is 'derived from first principles, looking at the software and its packaging definition' by reimplementing each ecosystem's version-resolution algorithm, explicitly 'without actually building and installing the software'.",
  "what_it_updates": "A dependency graph 'decorated with versioning and licensing information, known vulnerabilities, and other important signals of code health and safety'. Freshness comes from update feeds plus a constant-rate background rescan of every known package.",
  "loop_evidence": "unknown",
  "cold_start_approach": "not_applicable — static reconstruction, no learning phase.",
  "privacy_approach": "not_applicable — public package data.",
  "sparse_data_fit": "not_applicable as a learning mechanism; it does not learn. Recorded as a denominator boundary: the most authoritative package-health infrastructure at Google scale still contains no outcome loop, which is evidence about the market rather than a mechanism to copy.",
  "limitations": "No scoring formula, no OpenSSF Scorecard integration, and no production telemetry documented on the FAQ. Validation is correctness-checking against native tooling (agreement '99% or higher'), which is accuracy against a reference implementation, not outcome-based learning. Covers Cargo, Go, Maven, npm, NuGet, PyPI, RubyGems; C/C++ excluded for lack of a packaging model. The underlying platform is not open source, limiting methodology transparency.",
  "disposition": "census",
  "top10_rank": null,
  "top10_rationale": null
}

{
  "id": "P15-C-012",
  "name": "GrowthBook Bandits",
  "category": "bandits",
  "source_url": "https://docs.growthbook.io/bandits/config",
  "observed_date": "2026-08-27",
  "evidence_class": "first_party_docs",
  "claim": "Thompson-sampling bandit that publishes explicit hard minimums before it will reweight at all — the most concrete sparse-data threshold found in this census.",
  "feedback_signals": "A single Decision Metric per bandit, measured per variation over the update window.",
  "what_it_updates": "Traffic allocation weights across variations, recalculated on an operator-set cadence; allocation is 'proportionally to the probability that an arm is best'.",
  "loop_evidence": "documented_methodology",
  "cold_start_approach": "An explicit Exploration Window during which weights are frozen, gated by a hard requirement: 'We require at least 100 users per variation before the bandit updates variation weights', with a recommendation of 'at least 40 conversions per variation before you begin updating your bandit' (worked example: ~400 units per variation at a 10% conversion rate). Docs also warn that very short windows skew by timezone.",
  "privacy_approach": "unknown",
  "sparse_data_fit": "FAILS, and its own documentation is the proof. 100 users per arm before any weight update, 40 conversions per arm recommended — against a shelf asset that may appear in three builds total, this mechanism cannot legally fire. Recorded precisely because the number is published: it is the cleanest citable evidence that per-asset bandits are not transferable to a tens-of-builds regime. Two design elements DO transfer regardless of volume: the 1% traffic floor ('GrowthBook ensures that each variation has at least 1% of traffic, in case your user behavior changes over time'), which prevents an arm from being permanently starved and keeps a shelf asset re-testable after a bad early run; and the recommendation to prefer a daily-or-longer cadence because longer windows 'reduce the likelihood that a fluky day of traffic will cause undesirable weight updates' — the same anti-jitter argument applies with far more force at ten observations than at ten thousand.",
  "limitations": "Multi-armed only — GrowthBook 'does not currently offer contextual bandits that use per-user attributes', so it cannot condition on client context at all. Requires the Bayesian engine (frequentist is a flat no). Documented as best for 5+ variations. Minimum settable cadence is 15 minutes but real pipeline latency makes that unrealistic. Without sticky bucketing, returning users can be reassigned as weights shift.",
  "disposition": "top10",
  "top10_rank": 7,
  "top10_rationale": "Earns a top-10 slot as the census's most useful negative result plus two survivable positives. The published floors — 100 users and ~40 conversions per arm — are the number to quote to Cena when explaining why Actionist cannot run a per-asset bandit over a shelf at tens of builds; this is a vendor stating its own mechanism's data requirement, not an outside estimate. What survives the sparsity test is the 1% floor, which guarantees no asset is permanently written off by an unlucky first build, and the explicit preference for slow update cadences to avoid reacting to noise. Both are cheap to implement and both matter more, not less, when evidence is thin."
}

{
  "id": "P15-C-013",
  "name": "Braintrust human review",
  "category": "ml_eval",
  "source_url": "https://www.braintrust.dev/docs/guides/human-review",
  "observed_date": "2026-08-27",
  "evidence_class": "first_party_docs",
  "claim": "Structured human-review scoring on spans, feeding dataset curation and eval — but every promotion step is human-initiated; no automatic loop is claimed.",
  "feedback_signals": "Configurable score types on spans: categorical (predefined options mapped 0-100%), continuous (slider numeric), and free-form text written to metadata or expected.<path>. Production end-user feedback flows into the same pool.",
  "what_it_updates": "Nothing automatically. Reviewed logs are promoted into eval datasets by a human; evaluators are authored by a human; the single automated affordance is playground 'prompt improvement suggestions based on your annotations', which still requires a person to accept them. Dashboards let you 'monitor user satisfaction trends and correlate automated scores with user feedback' — an observation surface, not an actuator.",
  "loop_evidence": "documented_methodology",
  "cold_start_approach": "not_applicable — no model is being fit, so no prior is needed.",
  "privacy_approach": "Score visibility can be restricted by member or permission group with SQL-based conditional display, but docs are explicit that this is a display filter and not a security boundary — reviewers can reveal hidden scores via a 'Show all scores' toggle.",
  "sparse_data_fit": "STRONG, because it deliberately does not learn statistically. Structured human judgement on a handful of traces is exactly the regime Actionist is in: at tens of builds, a reviewer's categorical verdict on why a plan was edited carries far more information than any aggregate a statistical loop could extract. The transferable design is capturing edits as typed, queryable scores against a schema rather than as free text, so that ten observations are still analysable.",
  "limitations": "The loop is human-in-the-loop by design and does not close on its own — labelling it an automated production-learning system would be wrong. Model selection is not mentioned at all. Human review scorers are gated to Pro and Enterprise tiers. Editing a score changes future behaviour only; values already written to traces stay as they were.",
  "disposition": "top10",
  "top10_rank": 8,
  "top10_rationale": "Included because it represents the design that actually fits Actionist's data regime, and it is honest about being manual. At tens of builds the correct mechanism for learning from client edit patterns is not a bandit but a typed schema for recording what was edited and why, queryable later — which is exactly what Braintrust ships. The specific lesson is that structured scores beat free-text notes: a categorical field mapped to a numeric range gives you something analysable at n=10, whereas prose does not. Its explicit warning that score visibility is a display filter and not a security boundary is also directly relevant to P15(e), since it marks the difference between hiding cross-client data in the UI and actually isolating it."
}

{
  "id": "P15-C-014",
  "name": "LangSmith online evaluation",
  "category": "ml_eval",
  "source_url": "https://docs.langchain.com/langsmith/evaluation",
  "observed_date": "2026-08-27",
  "evidence_class": "first_party_docs",
  "claim": "Evaluators run automatically on sampled production traces, but the resulting dataset, evaluator and prompt updates are all documented as manual human actions.",
  "feedback_signals": "Online evaluator scores computed on live production traces (safety checks, format validation, quality heuristics, reference-free LLM-as-judge), plus human review. Sampling rates and filters control cost.",
  "what_it_updates": "Automatically: only scores attached to runs or threads, enabling 'real-time monitoring, anomaly detection, and alerting'. Manually: datasets (a human adds failing production traces), evaluators (a human authors targeted ones), and application versions (validated by offline experiment before redeploy).",
  "loop_evidence": "documented_methodology",
  "cold_start_approach": "unknown — datasets can be seeded from 'manually curated test cases, historical production traces, or synthetic data generation', the last of which is a documented way to bootstrap before real traffic exists.",
  "privacy_approach": "unknown",
  "sparse_data_fit": "MODERATE. The regression-capture pattern — 'Add real traces to a dataset so a failure you saw once becomes a test you run every time' — works at n=1 by construction and is the most sparse-appropriate learning mechanism in the whole ml_eval tier: one observed failure becomes a permanent guard. Synthetic dataset seeding is also a legitimate cold-start answer when real builds are too few. What does not transfer is anything aggregate, since sampled scoring assumes traffic volume Actionist will not have.",
  "limitations": "No automatic prompt optimization from feedback is documented on this page, and end-user feedback capture (thumbs up/down, ratings) is not documented here as a distinct input. Every loop-closing step requires a human. Page reached via 308 redirect from docs.smith.langchain.com/evaluation/concepts.",
  "disposition": "census",
  "top10_rank": null,
  "top10_rationale": null
}

{
  "id": "P15-C-015",
  "name": "WordPress.org plugin directory search",
  "category": "marketplace_ranking",
  "source_url": "https://developer.wordpress.org/plugins/wordpress-org/plugin-developer-faq/",
  "observed_date": "2026-08-27",
  "evidence_class": "first_party_docs",
  "claim": "Ranking algorithm is not published; the FAQ confirms usage data is a factor only indirectly, via an explicit statement that new plugins rank poorly because no usage data exists yet.",
  "feedback_signals": "Not enumerated. The FAQ's advice for improving rank is 'Write a good readme for the language, answer support posts promptly, get good reviews' — implying readme quality, support responsiveness and reviews matter, with no weights given.",
  "what_it_updates": "Position in plugin directory search results.",
  "loop_evidence": "vendor_claim",
  "cold_start_approach": "Documented only as a deficit, which is itself the useful finding: for a new plugin 'we have no data on usage, so you may need to wait a bit'. New plugins take 'usually 6 to 14 days after a plugin is committed to SVN' to appear in search at all. No prior or fallback is described — the plugin simply ranks badly until evidence accumulates.",
  "privacy_approach": "unknown",
  "sparse_data_fit": "FAILS as a model to copy, and is recorded as the anti-pattern. A new asset with no usage history is left to rank badly with no substitute signal, which for Actionist would mean every newly added shelf capability is invisible precisely when it most needs trial. This is the exact failure mode that Personalizer's Apprentice mode and Renovate's Neutral state avoid, and it argues that Actionist must define a deliberate cold-start prior rather than letting absence of evidence read as evidence of absence.",
  "limitations": "No algorithm, no weights, no confirmation of active installs, star ratings, support-thread resolution rate or update recency as ranking factors. Tag handling is the only mechanical detail given: readmes cap at 12 tags, only the first five display, 'The first 12 tags are used for searches, and the rest are ignored, so tag-stuffing won't help you at all'. Everything about ranking here is advice rather than methodology.",
  "disposition": "census",
  "top10_rank": null,
  "top10_rationale": null
}

{
  "id": "P15-C-016",
  "name": "VS Code Marketplace",
  "category": "marketplace_ranking",
  "source_url": "https://code.visualstudio.com/api/references/extension-manifest",
  "observed_date": "2026-08-27",
  "evidence_class": "first_party_docs",
  "claim": "No published ranking methodology; first-party extension documentation covers only which manifest fields feed text search and filtering, not how results are ordered.",
  "feedback_signals": "None documented. Discoverability guidance is limited to author-supplied metadata: keywords ('currently limited to 30 keywords'), display name and description which 'are also used for text search in VS Code', and categories which group extensions 'on the Marketplace which improves filtering and discovery'.",
  "what_it_updates": "unknown — the document describes inputs to text search and filtering, not result ordering or weighting.",
  "loop_evidence": "unknown",
  "cold_start_approach": "unknown",
  "privacy_approach": "unknown",
  "sparse_data_fit": "not_applicable — no mechanism is disclosed to evaluate.",
  "limitations": "Install counts, star ratings, trending computation and any ranking formula are entirely absent from first-party extension-authoring docs. Recorded as a denominator entry and a negative: one of the largest software-component marketplaces publishes nothing about outcome-driven ranking, reinforcing that marketplace ranking methodology is generally trade-secret and therefore a weak evidence base for P15.",
  "disposition": "excluded",
  "top10_rank": null,
  "top10_rationale": null
}

{
  "id": "P15-C-017",
  "name": "Amazon Personalize User-Personalization",
  "category": "ranking_service",
  "source_url": "https://docs.aws.amazon.com/personalize/latest/dg/native-recipe-new-item-USER_PERSONALIZATION.html",
  "observed_date": "2026-08-27",
  "evidence_class": "first_party_docs",
  "claim": "Managed recommender with an explicit, parameterised cold-start exploration mechanism: exploration_weight and exploration_item_age_cut_off control how often low-evidence items are surfaced.",
  "feedback_signals": "Item interaction events (click, watch, purchase) with event type and optional event value; contextual metadata (location, device type); and impressions data — the list of items visible to the user when they interacted with one, which lets the model distinguish 'not chosen' from 'not shown'.",
  "what_it_updates": "A trained solution version (ranking model). Documented automatic updates every two hours to incorporate new items; full retraining is a separate explicit trainingMode='FULL' operation.",
  "loop_evidence": "documented_methodology",
  "cold_start_approach": "The most concretely parameterised cold-start in this census. exploration_weight (default 0.3, range [0.0,1.0]) sets how frequently recommendations include items with less interaction data — 'At zero, no exploration occurs and recommendations are based on current data (relevance)'. exploration_item_age_cut_off (default 30.0 days, minimum 1) bounds which items count as new enough to explore, with age taken from creation timestamp or, failing that, interaction data. recency_mask (default True) additionally weights recent events more heavily.",
  "privacy_approach": "unknown",
  "sparse_data_fit": "FAILS at the model level, transfers at the parameter level. AWS requires importing 'at minimum 1000 item interactions' before the recipe can be used at all — a hard floor Actionist will not clear per asset, and probably not in total for years. But two ideas survive independently of volume. First, impressions data: recording which shelf assets were SHOWN in a candidate plan and not chosen, distinct from never being considered, roughly triples the information extractable from a single build and is free to log. Second, an explicit exploration budget expressed as a fraction (0.3) combined with an age cutoff (30 days) is a policy, not a learned quantity — Actionist can set 'reserve ~30% of candidate slots for assets newer than N days' by hand on day one and it will behave sensibly at ten builds.",
  "limitations": "Hard 1000-interaction minimum to train. Superseded by User-Personalization-v2, which AWS explicitly recommends instead. Two-hour auto-update only considers new items; genuine model improvement needs full retraining. Docs note recommendations may include items older than the exploration cutoff when they are relevant anyway, so the cutoff bounds exploration, not eligibility.",
  "disposition": "top10",
  "top10_rank": 4,
  "top10_rationale": "Supplies the best-specified answer to P15(c) and (d) as a pair, in a form that is a hand-settable policy rather than a learned parameter. exploration_weight = 0.3 with a 30-day age cutoff is a concrete, citable default for 'how much of the shelf ranking should be reserved for assets that have not yet earned their place', and unlike a bandit's posterior it needs no data to initialise. The impressions concept is the sleeper finding of this census: at tens of builds, the difference between an asset that was considered and rejected and one that was never surfaced is a large fraction of the total available signal, and most systems throw it away. Ranked fourth because the surrounding model is unusable at Actionist's scale — the 1000-interaction floor is disqualifying — but the two mechanisms above are directly implementable."
}

{
  "id": "P15-C-018",
  "name": "ISRG Divvi Up (DAP / Prio)",
  "category": "privacy_aggregation",
  "source_url": "https://divviup.org/about/",
  "observed_date": "2026-08-27",
  "evidence_class": "first_party_docs",
  "claim": "Nonprofit-operated privacy-preserving telemetry: each client measurement is split into two encrypted secret shares sent to non-colluding aggregators, which each compute partial sums that are recombined into an aggregate.",
  "feedback_signals": "Arbitrary client-side metrics, split 'into two encrypted shares as it leaves the origin'.",
  "what_it_updates": "Aggregate statistics or averages over the whole population; no per-client record is ever reconstructible by either aggregator alone.",
  "loop_evidence": "documented_methodology",
  "cold_start_approach": "not_applicable — an aggregation transport, not a learner.",
  "privacy_approach": "Two-party secret sharing with a strict non-collusion assumption: the two data share processors 'do not share data with each other', so 'only minimal information about the original data is revealed to either processor'. The second aggregator must be operated independently — by your own organisation or another provider. Implements IETF DAP over VDAFs; Janus is the Rust implementation; lineage traces to Prio (Corrigan-Gibbs and Boneh, Stanford). Operated by ISRG, the nonprofit behind Let's Encrypt.",
  "sparse_data_fit": "FAILS decisively, and this is the most important negative finding for P15(e). DAP mandates a min_batch_size — 'the smallest number of reports a batch is allowed to include' — and the specification states plainly that for reasonable privacy 'there must be a large number of Clients'. With tens of builds across a handful of clients, no batch would ever reach a threshold that provides real protection, and setting the threshold low enough to fire would defeat the guarantee. Cryptographic cross-client aggregation is therefore the wrong tool at Actionist's scale. The correct sparse-regime answer is contractual and architectural isolation plus human review of what crosses a client boundary, not a DAP deployment.",
  "limitations": "Requires standing up or contracting a second independent aggregator — real operational cost and a governance relationship, not a library import. The non-collusion assumption is the whole security model. The about page does not quantify minimum batch size or what a single aggregator can infer beyond 'minimal information'; the min_batch_size requirement was confirmed separately from the IETF DAP draft (draft-ietf-ppm-dap, TaskConfiguration: 'uint64 min_batch_size').",
  "disposition": "top10",
  "top10_rank": 9,
  "top10_rationale": "Earns a slot as the definitive, citable reason NOT to build cryptographic cross-client aggregation for Actionist. P15(e) asks how to aggregate privacy-safely across clients, and the mature standards-track answer to that question requires a large client population by construction — DAP's own min_batch_size field and the draft's statement that privacy needs many clients settle it. Recording this prevents an expensive wrong turn: the temptation to reach for Prio-style aggregation because it sounds principled, when at tens of builds it either cannot fire or fires without protecting anything. The genuinely transferable idea is the trust structure rather than the cryptography — designate who may see cross-client data and enforce that no single party sees both the raw signal and the client identity."
}

{
  "id": "P15-C-019",
  "name": "Apple local differential privacy telemetry",
  "category": "privacy_aggregation",
  "source_url": "https://www.apple.com/privacy/docs/Differential_Privacy_Overview.pdf",
  "observed_date": "2026-08-27",
  "evidence_class": "first_party_docs",
  "claim": "Publishes per-feature epsilon values and hard per-user daily donation caps for local differential privacy applied on-device before transmission.",
  "feedback_signals": "Per-feature usage events privatised on device: QuickType suggestions, emoji selection, Lookup Hints, Safari energy-draining domains, Safari autoplay intent, Safari crashing domains, Health data-type usage (which types are edited, not the health data itself).",
  "what_it_updates": "Aggregate population statistics — e.g. trending words, most-used emoji, problematic domains — shared with internal Apple teams to improve those features.",
  "loop_evidence": "documented_methodology",
  "cold_start_approach": "not_applicable.",
  "privacy_approach": "Local DP: noise is added on-device so 'Apple's servers don't receive clear data'. Published parameters: Lookup Hints epsilon 4, capped at two donations per user per day; emoji epsilon 4, one donation per day; QuickType epsilon 8, two per day; Health types epsilon 2, one per day; Safari energy/crashing domains epsilon 4 on a single shared budget, Safari autoplay epsilon 8; Safari capped at 2 donations per day overall. Device identifiers removed, IP addresses dropped at ingestion and not stored, data retained at most three months, opt-in. Count Mean Sketch encodes input via SHA-256-derived hashes into a fixed-size matrix, flipping each vector coordinate with probability 1/(1+e^(epsilon/2)).",
  "sparse_data_fit": "FAILS, by the same structural argument as DAP and stated more plainly in Apple's own text: the mechanism works because 'the noise that has been added can average out over large numbers of data points'. At tens of observations the noise does not average out — it dominates, and the aggregate is meaningless. Recorded as the second independent confirmation that local DP is unavailable to Actionist. What does transfer is the per-source contribution cap: limiting how many observations any single client can contribute to a shared statistic prevents one large client from silently determining the whole shelf ranking. That is a fairness and robustness control that works at any scale, and is arguably more valuable to Actionist than the privacy property it was invented for.",
  "limitations": "Requires population scale to be useful at all. Epsilon 8 for QuickType with two donations daily is a weak formal guarantee by academic standards and has been criticised as such; the effective per-day budget is the product of epsilon and the donation cap. Values quoted are from Apple's own overview PDF and reflect the features listed there (through iOS 11 / macOS High Sierra era); they may not describe current deployments.",
  "disposition": "census",
  "top10_rank": null,
  "top10_rationale": null
}

{
  "id": "P15-C-020",
  "name": "Vercel v0 composite model family / vercel-autofixer-01",
  "category": "builder_loop",
  "source_url": "https://vercel.com/blog/v0-composite-model-family",
  "observed_date": "2026-08-27",
  "evidence_class": "first_party_docs",
  "claim": "The only AI app-builder found with a first-party documented production learning loop: user feedback and evals identify recurring generation error categories, and a custom model is reinforcement-fine-tuned to drive those error rates down.",
  "feedback_signals": "Two documented inputs: 'a comprehensive set of evals, along with feedback from v0.dev users', used to identify categories of error the base model repeatedly makes (over-formatting with markdown, misplaced files, subtle bugs). The headline product metric is the percentage of successful generations — one that produces a working site in preview rather than an error or blank screen.",
  "what_it_updates": "vercel-autofixer-01, a small fast model trained 'in conjunction with Fireworks AI using reinforcement fine-tuning (RFT)' over multiple iterations to 'minimize error rates across a variety of tracked categories'. It inspects the base model's output stream mid-generation, with a post-stream pass and a linter run. The composite architecture is deliberately modular so the frontier LLM can be swapped without disturbing the rest.",
  "loop_evidence": "documented_methodology",
  "cold_start_approach": "Retrieval-augmented generation supplies current domain knowledge rather than requiring the model to have learned it, explicitly because 'Model knowledge can quickly become outdated for topics that change fast'. This is a knowledge-injection answer to staleness rather than a statistical prior.",
  "privacy_approach": "Per v0's FAQ (secondary to this page): user prompts and content may be used as inputs to models and learning systems to improve products, but Enterprise customer content is not used for training, and customer data or code from Vercel platform services is not used to train or fine-tune v0's models. Tiered consent — the paying enterprise tier is excluded from the loop.",
  "sparse_data_fit": "PARTIAL, and the distinction matters. The RFT training itself needs volume Actionist does not have — Vercel describes 'a large volume of real generations'. But the loop's architecture is the transferable part and it is sparsity-tolerant: errors are bucketed into named recurring CATEGORIES first, and only then is anything trained. Categorising ten failures into three named failure modes is entirely feasible and immediately actionable; it converts sparse observations into a structured taxonomy that a human or a prompt change can act on, without needing a model update at all. Actionist should take the taxonomy discipline and the error-rate-as-primary-metric framing, and skip the RFT.",
  "limitations": "Crucially, Vercel does NOT state that the model is trained on accept/reject labels — the blog documents how error patterns were IDENTIFIED (evals plus user feedback) but 'does not specify the training dataset itself'. Treating this as documented accept/reject learning would overclaim. Error-free generation rates quoted (v0-1.5-md 93.87, v0-1.5-lg 89.80) are Vercel's own eval sets, self-reported and not independently verified. The post also flags the larger model scoring lower than the medium one.",
  "disposition": "top10",
  "top10_rank": 3,
  "top10_rationale": "The closest commercial analogue to Actionist's own business — a system that generates application code and improves from what happens to those generations in production — and the only one in tier 8 with substantive first-party documentation rather than marketing. Two things transfer directly and neither needs volume. First, defining a single unambiguous production success metric: 'does the generated thing actually work in preview' is binary, observable per build, and meaningful at n=1, which is exactly what a sparse regime needs and what vaguer satisfaction metrics fail to provide. Second, bucketing failures into named recurring categories before attempting any learning, so that ten observations become a taxonomy rather than noise. Ranked third behind Renovate and Personalizer because its actual learning step (RFT on a large generation corpus) is out of reach, but its measurement discipline is adoptable this quarter."
}

{
  "id": "P15-C-021",
  "name": "Cursor Tab",
  "category": "builder_loop",
  "source_url": "https://cursor.com/docs",
  "observed_date": "2026-08-27",
  "evidence_class": "first_party_docs",
  "claim": "No first-party documentation found for learning from accepted versus rejected suggestions; the Tab model is not described in the public docs landing page or model table.",
  "feedback_signals": "unknown — not documented on the pages reached.",
  "what_it_updates": "unknown.",
  "loop_evidence": "unknown",
  "cold_start_approach": "unknown",
  "privacy_approach": "Not established for Cursor itself. Privacy Mode appears only once on the docs landing page, in a per-model data-retention note stating a model 'Requires data retention approval for Enterprise customers, Teams and individual customers with Privacy Mode enabled', with the accompanying non-training statement being Anthropic's policy for that model rather than a description of Cursor's own handling of user code.",
  "sparse_data_fit": "not_applicable — no mechanism disclosed to evaluate.",
  "limitations": "docs.cursor.com/tab/overview 308-redirects to cursor.com/docs, which is a landing page whose model table covers agent/chat models only and lists no Tab or autocomplete model and no training methodology. Widely assumed in the industry to learn from accept/reject telemetry, but that assumption is NOT supported by first-party documentation observed in this run. Recorded as unknown rather than inferred. A dedicated privacy or security page likely holds the training-policy detail but was not reached before this run's scope closed.",
  "disposition": "census",
  "top10_rank": null,
  "top10_rationale": null
}

{
  "id": "P15-C-022",
  "name": "Lovable / Replit Agent (builder learning-loop claims)",
  "category": "builder_loop",
  "source_url": "https://vercel.com/blog/how-we-made-v0-an-effective-coding-agent",
  "observed_date": "2026-08-27",
  "evidence_class": "secondary",
  "claim": "No official documentation located from Lovable or Replit describing an accepted/rejected-generation training loop; available coverage is comparative review material, not vendor methodology.",
  "feedback_signals": "unknown. Replit Agent is described in third-party coverage as reading errors and fixing its own code, which is in-session self-correction — a within-run repair loop, not a cross-build learning loop that updates any persistent model or ranking.",
  "what_it_updates": "unknown; no evidence of persistent cross-session learning.",
  "loop_evidence": "unknown",
  "cold_start_approach": "unknown",
  "privacy_approach": "unknown",
  "sparse_data_fit": "not_applicable — no disclosed mechanism.",
  "limitations": "This record is a deliberate NEGATIVE denominator entry and should not be read as a finding about these products' actual internals, only about what they publish. Search surfaced only comparative/review sources (uibakery.io, altar.io, designrevision.com) rather than vendor documentation. The source_url attached is the Vercel engineering post used as the contrast case, since no Lovable or Replit first-party methodology page was located; this is the weakest-sourced record in the set and is marked secondary accordingly. The important distinction it preserves: in-session self-repair is common and easy to mistake for production learning, but it teaches the system nothing that survives the session.",
  "disposition": "excluded",
  "top10_rank": null,
  "top10_rationale": null
}