Current synthesis from the research packet. Exact records and complete source artifacts remain visible below.
Define the evidence flywheel and ranking model using adaptation effort, workflow success, runtime burden and production history.
[private local source withheld]research/actionmodel-builder-research-2026-08-26/phase-3/outputs/decision-ledger.mdknowledge/00-MASTER-SYNTHESIS.mdTheir thousandth app is generated by the same model against the same blank context as their first, so nothing accumulates except model upgrades everyone else also receives. Attaching outcome, adaptation-cost and incident evidence to stable reusable units makes every build rank the shelf for the next one — the advantage a pure generator cannot copy, because it keeps no unit for the learning to attach to.
22 mined records. Search and inspect the full agent output; nothing is collapsed into a top-ten headline.
| Rank | Identity | What the agent found | Evidence | Source |
|---|---|---|---|---|
| 2 | Azure AI Personalizerbandits | Managed contextual-bandit service exposing a two-call Rank/Reward contract; retiring 25 August 2026 with migration recommended to open-source microsoft/learning-loop. | first_party_docstop10 | Source ↗ |
| 10 | Algolia Dynamic Re-Rankingranking_service | Reranks an already-eligible result set using click and conversion events over a sliding window, with published minimum-evidence thresholds before any record is promoted. | first_party_docstop10 | Source ↗ |
| — | Statsig Autotunebandits | Multi-armed bandit that allocates traffic across variants in proportion to each variant's posterior probability of being best, using Thompson sampling. | first_party_docscensus | Source ↗ |
| — | Eppo Contextual Banditsbandits | Contextual bandit that predicts the objective metric per action from subject/action context, then selects with injected randomness, retraining on a scheduled cadence. | first_party_docscensus | Source ↗ |
| — | Shopify App Store search rankingmarketplace_ranking | Marketplace ranking for reusable software components where merchant-side outcome signals, not just metadata, determine placement; Shopify first-party docs confirm a recency- and trust-weighted rating feeds search placement. | first_party_docscensus | Source ↗ |
| 1 | Mend Renovate Merge Confidenceregistry_health | Attaches Age/Adoption/Passing/Confidence badges to dependency-update PRs, computed from aggregated real-world CI outcomes across the hosted Renovate fleet, in order to 'find and flag undeclared breaking releases'. | first_party_docstop10 | Source ↗ |
| 6 | Socket.dev package scoresregistry_health | Publishes the actual scoring formula, including per-severity normalization functions, soft caps, and a power-scaling exponent gamma derived from project size and popularity. | first_party_docstop10 | Source ↗ |
| — | ecosyste.msregistry_health | Exposes per-package percentile rankings across real adoption signals via public API, alongside raw dependent and download counts and a boolean criticality flag. | observed_behaviorcensus | Source ↗ |
| 5 | npms.io scoring (npms-analyzer)registry_health | Published an unusually explicit normalization method: each package's raw evaluation is normalized against the corpus min/max, then mapped through a Bezier curve anchored on the corpus mean. | first_party_docstop10 | Source ↗ |
| — | OpenSSF Scorecardregistry_health | Scores repositories 0-10 per check and aggregates by risk weight, entirely from static repository and project metadata — no downstream or production usage signal enters the score. | first_party_docscensus | Source ↗ |
| — | deps.dev (Google Open Source Insights)registry_health | Builds annotated dependency graphs from first principles without building or installing software; provides no scoring system and no production feedback loop. | first_party_docscensus | Source ↗ |
| 7 | GrowthBook Banditsbandits | Thompson-sampling bandit that publishes explicit hard minimums before it will reweight at all — the most concrete sparse-data threshold found in this census. | first_party_docstop10 | Source ↗ |
| 8 | Braintrust human reviewml_eval | Structured human-review scoring on spans, feeding dataset curation and eval — but every promotion step is human-initiated; no automatic loop is claimed. | first_party_docstop10 | Source ↗ |
| — | LangSmith online evaluationml_eval | Evaluators run automatically on sampled production traces, but the resulting dataset, evaluator and prompt updates are all documented as manual human actions. | first_party_docscensus | Source ↗ |
| — | WordPress.org plugin directory searchmarketplace_ranking | Ranking algorithm is not published; the FAQ confirms usage data is a factor only indirectly, via an explicit statement that new plugins rank poorly because no usage data exists yet. | first_party_docscensus | Source ↗ |
| — | VS Code Marketplacemarketplace_ranking | No published ranking methodology; first-party extension documentation covers only which manifest fields feed text search and filtering, not how results are ordered. | first_party_docsexcluded | Source ↗ |
| 4 | Amazon Personalize User-Personalizationranking_service | Managed recommender with an explicit, parameterised cold-start exploration mechanism: exploration_weight and exploration_item_age_cut_off control how often low-evidence items are surfaced. | first_party_docstop10 | Source ↗ |
| 9 | ISRG Divvi Up (DAP / Prio)privacy_aggregation | Nonprofit-operated privacy-preserving telemetry: each client measurement is split into two encrypted secret shares sent to non-colluding aggregators, which each compute partial sums that are recombined into an aggregate. | first_party_docstop10 | Source ↗ |
| — | Apple local differential privacy telemetryprivacy_aggregation | Publishes per-feature epsilon values and hard per-user daily donation caps for local differential privacy applied on-device before transmission. | first_party_docscensus | Source ↗ |
| 3 | Vercel v0 composite model family / vercel-autofixer-01builder_loop | The only AI app-builder found with a first-party documented production learning loop: user feedback and evals identify recurring generation error categories, and a custom model is reinforcement-fine-tuned to drive those error rates down. | first_party_docstop10 | Source ↗ |
| — | Cursor Tabbuilder_loop | No first-party documentation found for learning from accepted versus rejected suggestions; the Tab model is not described in the public docs landing page or model table. | first_party_docscensus | Source ↗ |
| — | Lovable / Replit Agent (builder learning-loop claims)builder_loop | No official documentation located from Lovable or Replit describing an accepted/rejected-generation training loop; available coverage is comparative review material, not vendor methodology. | secondaryexcluded | Source ↗ |
97 mined records. Search and inspect the full agent output; nothing is collapsed into a top-ten headline.
| Rank | Identity | What the agent found | Evidence | Source |
|---|---|---|---|---|
| 1 | VowpalWabbit/vowpal_wabbitbandits | Vowpal Wabbit is a machine learning system which pushes the frontier of machine learning with techniques such as online, hashing, allreduce, reductions, learning2search, active, and interactive learning. | first_party_docstop10 | Source ↗ |
| 3 | fidelity/mabwiserbandits | MABWiser: Contextual Multi-Armed Bandits Library | first_party_docstop10 | Source ↗ |
| — | david-cortes/contextualbanditsbandits | Python implementations of contextual bandits algorithms | first_party_docscensus | Source ↗ |
| 7 | st-tech/zr-obpbandits | Open Bandit Pipeline: a python library for bandit algorithms and off-policy evaluation | first_party_docstop10 | Source ↗ |
| — | ray-project/raybandits | Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads. | observed_behaviorcensus | Source ↗ |
| — | tensorflow/agentsbandits | TF-Agents: A reliable, scalable and easy to use TensorFlow library for Contextual Bandits and Reinforcement Learning. | observed_behaviorcensus | Source ↗ |
| — | motiwari/BanditPAMbandits | BanditPAM C++ implementation and Python package | observed_behaviorexcluded | Source ↗ |
| — | SMPyBandits/SMPyBanditsbandits | 🔬 Research Framework for Single and Multi-Players 🎰 Multi-Arms Bandits (MAB) Algorithms, implementing all the state-of-the-art algorithms for single-player (UCB, KL-UCB, Thompson...) and multi-player (MusicalChair, MEGA, rhoRand, MCTop/R... | observed_behaviorcensus | Source ↗ |
| — | bgalbraith/banditsbandits | Python library for Multi-Armed Bandits | observed_behaviorexcluded | Source ↗ |
| — | johnmyleswhite/BanditsBookbandits | Code for my book on Multi-Armed Bandit Algorithms | secondaryexcluded | Source ↗ |
| — | alison-carrera/onnbandits | Online Deep Learning: Learning Deep Neural Networks on the Fly / Non-linear Contextual Bandit Algorithm (ONN_THS) | observed_behaviorcensus | Source ↗ |
| 2 | meta-pytorch/botorchbayesian_preference | Bayesian optimization in PyTorch | first_party_docstop10 | Source ↗ |
| 4 | facebook/Axbayesian_preference | Adaptive Experimentation Platform | first_party_docstop10 | Source ↗ |
| 5 | lucasmaystre/choixbayesian_preference | Inference algorithms for models based on Luce's choice axiom | first_party_docstop10 | Source ↗ |
| 6 | pymc-devs/pymcbayesian_preference | Bayesian Modeling and Probabilistic Programming in Python | first_party_docstop10 | Source ↗ |
| — | stan-dev/stanbayesian_preference | Stan development repository. The master branch contains the current release. The develop branch contains the latest stable development. See the Developer Process Wiki for details. | observed_behaviorcensus | Source ↗ |
| — | stan-dev/cmdstanpybayesian_preference | CmdStanPy is a lightweight interface to Stan for Python users which provides the necessary objects and functions to compile a Stan program and fit the model to data using CmdStan. | observed_behaviorcensus | Source ↗ |
| — | scikit-optimize/scikit-optimizebayesian_preference | Sequential model-based optimization with a `scipy.optimize` interface | observed_behaviorexcluded | Source ↗ |
| — | optuna/optunabayesian_preference | A hyperparameter optimization framework | observed_behaviorcensus | Source ↗ |
| — | yuki-koyama/sequential-gallerybayesian_preference | Sequential Gallery for Interactive Visual Design Optimization [SIGGRAPH 2020] | observed_behaviorcensus | Source ↗ |
| — | sublee/trueskillbayesian_preference | An implementation of the TrueSkill rating system for Python | first_party_docsexcluded | Source ↗ |
| — | philihp/openskill.jsbayesian_preference | A faster, open-license alternative to Microsoft TrueSkill | observed_behaviorcensus | Source ↗ |
| — | hyperopt/hyperoptbayesian_preference | Distributed Asynchronous Hyperparameter Optimization in Python | observed_behaviorcensus | Source ↗ |
| — | automl/SMAC3bayesian_preference | SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization | observed_behaviorcensus | Source ↗ |
| 10 | growthbook/growthbookexperimentation | Open Source Feature Flags, Experimentation, and Product Analytics | first_party_docstop10 | Source ↗ |
| — | Unleash/unleashexperimentation | Open-source feature management platform | first_party_docscensus | Source ↗ |
| — | Flagsmith/flagsmithexperimentation | Flagsmith is an open source feature flagging and remote config service. Self-host or use our hosted version at https://app.flagsmith.com. | observed_behaviorcensus | Source ↗ |
| — | PostHog/posthogexperimentation | :hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to... | first_party_docscensus | Source ↗ |
| — | Eppo-exp/eppo-multiplatformexperimentation | SDKs and platform artifacts to support Eppo Flagging. | observed_behaviorcensus | Source ↗ |
| — | facebookarchive/planoutexperimentation | PlanOut is a library and interpreter for designing online experiments. | first_party_docscensus | Source ↗ |
| — | intuit/wasabiexperimentation | Wasabi A/B Testing service is an open source project that is no longer under active development or being supported | observed_behaviorexcluded | Source ↗ |
| — | thomaspoignant/go-feature-flagexperimentation | GO Feature Flag is a simple, complete and lightweight self-hosted cloud native feature flag solution 100% Open Source — built on OpenFeature 🎛️ | observed_behaviorcensus | Source ↗ |
| — | open-feature/specexperimentation | OpenFeature specification | observed_behaviorcensus | Source ↗ |
| — | langfuse/langfusellm_eval_loop | 🪢 Open source AI engineering platform: LLM evals, observability, metrics, prompt management, playground, datasets. Integrates with OpenTelemetry, LangChain, OpenAI SDK, LiteLLM, and more. 🍊YC W23 | first_party_docscensus | Source ↗ |
| — | Arize-ai/phoenixllm_eval_loop | AI Observability & Evaluation | first_party_docsexcluded | Source ↗ |
| — | promptfoo/promptfoollm_eval_loop | Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenA... | observed_behaviorcensus | Source ↗ |
| — | confident-ai/deepevalllm_eval_loop | The LLM Evaluation Framework | observed_behaviorcensus | Source ↗ |
| — | openai/evalsllm_eval_loop | Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks. | observed_behaviorcensus | Source ↗ |
| — | braintrustdata/autoevalsllm_eval_loop | AutoEvals is a tool for quickly and easily evaluating AI model outputs using best practices. | observed_behaviorcensus | Source ↗ |
| — | wandb/weavellm_eval_loop | Weave is a toolkit for developing AI-powered applications, built by Weights & Biases. | observed_behaviorcensus | Source ↗ |
| — | truera/trulensllm_eval_loop | Evaluation and Tracking for LLM Experiments and AI Agents | observed_behaviorcensus | Source ↗ |
| — | vibrantlabsai/ragasllm_eval_loop | Supercharge Your LLM Application Evaluations 🚀 | observed_behaviorcensus | Source ↗ |
| — | comet-ml/opikllm_eval_loop | Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards. | observed_behaviorcensus | Source ↗ |
| — | uptrain-ai/uptrainllm_eval_loop | UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform root cause analysis on failure cases an... | observed_behaviorcensus | Source ↗ |
| — | benfred/implicitrecommender | Fast Python Collaborative Filtering for Implicit Feedback Datasets | first_party_docscensus | Source ↗ |
| 9 | lyst/lightfmrecommender | A Python implementation of LightFM, a hybrid recommendation algorithm. | first_party_docstop10 | Source ↗ |
| — | NicolasHug/Surpriserecommender | A Python scikit for building and analyzing recommender systems | observed_behaviorcensus | Source ↗ |
| — | RUCAIBox/RecBolerecommender | A unified, comprehensive and efficient recommendation library | observed_behaviorcensus | Source ↗ |
| — | recommenders-team/recommendersrecommender | Best Practices on Recommendation Systems | observed_behaviorcensus | Source ↗ |
| — | gorse-io/gorserecommender | AI powered open source recommender system engine supports classical/LLM rankers and multimodal content via embedding | observed_behaviorcensus | Source ↗ |
| — | lenskit/lkpyrecommender | Python recommendation toolkit | first_party_docscensus | Source ↗ |
| — | tensorflow/recommendersrecommender | TensorFlow Recommenders is a library for building recommender system models using TensorFlow. | observed_behaviorcensus | Source ↗ |
| — | NVIDIA-Merlin/Merlinrecommender | NVIDIA Merlin is an open source library providing end-to-end GPU-accelerated recommender systems, from feature engineering and preprocessing to training deep learning models and running inference in production. | observed_behaviorexcluded | Source ↗ |
| — | PreferredAI/cornacrecommender | A Comparative Framework for Multimodal Recommender Systems | observed_behaviorcensus | Source ↗ |
| — | ossf/scorecardasset_scoring | OpenSSF Scorecard - Security health metrics for Open Source | first_party_docscensus | Source ↗ |
| — | ossf/criticality_scoreasset_scoring | Gives criticality score for an open source project | observed_behaviorcensus | Source ↗ |
| — | librariesio/libraries.ioasset_scoring | :books: The Open Source Discovery Service | first_party_docscensus | Source ↗ |
| — | chaoss/grimoirelabasset_scoring | GrimoireLab: platform for software development analytics and insights | first_party_docscensus | Source ↗ |
| — | chaoss/augurasset_scoring | The Augur project is no longer part of CHAOSS. Use CollectOSS instead! | secondaryexcluded | Source ↗ |
| — | ossf/allstarasset_scoring | GitHub App to set and enforce security policies | observed_behaviorcensus | Source ↗ |
| — | google/deps.devasset_scoring | Resources for the deps.dev API | first_party_docscensus | Source ↗ |
| — | google/osv-scannerasset_scoring | Vulnerability scanner written in Go which uses the data provided by https://osv.dev | observed_behaviorcensus | Source ↗ |
| — | chaoss/wg-metrics-modelsasset_scoring | Working Group for Metrics Model | observed_behaviorcensus | Source ↗ |
| — | npms-io/npms-analyzerregistry_ranking | The analyzer behind https://npms.io | first_party_docscensus | Source ↗ |
| — | rust-lang/crates.ioregistry_ranking | The Rust package registry | observed_behaviorcensus | Source ↗ |
| — | pypi/warehouseregistry_ranking | The Python Package Index | observed_behaviorcensus | Source ↗ |
| — | npm/cliregistry_ranking | the package manager for JavaScript | observed_behaviorcensus | Source ↗ |
| — | opendp/opendpprivacy_telemetry | The core library of differential privacy algorithms powering the OpenDP Project. | first_party_docscensus | Source ↗ |
| — | google/differential-privacyprivacy_telemetry | Google's differential privacy libraries. | first_party_docscensus | Source ↗ |
| — | opendp/smartnoise-sdkprivacy_telemetry | Tools and service for differentially private processing of tabular and relational data | observed_behaviorcensus | Source ↗ |
| — | google/rapporprivacy_telemetry | RAPPOR: Privacy-Preserving Reporting Algorithms | first_party_docscensus | Source ↗ |
| — | divviup/janusprivacy_telemetry | Experimental implementation of the Distributed Aggregation Protocol (DAP) specification. | observed_behaviorcensus | Source ↗ |
| — | divviup/libprio-rsprivacy_telemetry | Implementation of Prio in Rust. | observed_behaviorcensus | Source ↗ |
| — | divviup/prio-serverprivacy_telemetry | A Prio server implementation. | observed_behaviorexcluded | Source ↗ |
| — | plausible/analyticsprivacy_telemetry | Open source, privacy-first web analytics. Lightweight, cookie-free Google Analytics alternative. Self-hosted or cloud. | first_party_docscensus | Source ↗ |
| — | umami-software/umamiprivacy_telemetry | Umami is a privacy-first analytics platform. Traffic, campaigns, behavior, conversions, and revenue in one place — no cookies, no surveillance, self-hosted or in the cloud. | observed_behaviorcensus | Source ↗ |
| — | opendp/tumult-analyticsprivacy_telemetry | Tumult Analytics is a Python library for privately computing aggregate queries on tabular data. It is built atop the Tumult Core library. | observed_behaviorcensus | Source ↗ |
| — | IBM/differential-privacy-libraryprivacy_telemetry | Diffprivlib: The IBM Differential Privacy Library | observed_behaviorcensus | Source ↗ |
| — | meta-pytorch/opacusprivacy_telemetry | Training PyTorch models with differential privacy | observed_behaviorcensus | Source ↗ |
| — | evidentlyai/evidentlydrift_monitoring | Evidently is an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics. | first_party_docscensus | Source ↗ |
| — | whylabs/whylogsdrift_monitoring | An open-source data logging library for machine learning models and data pipelines. 📚 Provides visibility into data quality & model performance over time. 🛡️ Supports privacy-preserving data collection, ensuring safety & robustness. 📈 | observed_behaviorcensus | Source ↗ |
| — | fivetran/great_expectationsdrift_monitoring | Always know what to expect from your data. | observed_behaviorcensus | Source ↗ |
| — | deepchecks/deepchecksdrift_monitoring | Deepchecks: Tests for Continuous Validation of ML Models & Data. Deepchecks is a holistic open-source solution for all of your AI & ML validation needs, enabling to thoroughly test your data and models from research to production. | secondarycensus | Source ↗ |
| — | SeldonIO/alibi-detectdrift_monitoring | Algorithms for outlier, adversarial and drift detection | secondarycensus | Source ↗ |
| — | NannyML/nannymldrift_monitoring | nannyml: post-deployment data science in python | observed_behaviorcensus | Source ↗ |
| — | IFCA-Advanced-Computing/frourosdrift_monitoring | Frouros: an open-source Python library for drift detection in machine learning systems. | observed_behaviorcensus | Source ↗ |
| — | sodadata/soda-coredrift_monitoring | Data Contracts engine for the modern data stack. https://www.soda.io | observed_behaviorcensus | Source ↗ |
| — | huggingface/trledit_learning | Train transformer language models with reinforcement learning. | observed_behaviorcensus | Source ↗ |
| — | continuedev/continueedit_learning | open-source coding agent | observed_behaviorcensus | Source ↗ |
| — | dyad-sh/dyadedit_learning | Local, open-source AI app builder for power users ✨ v0 / Lovable / Replit / Bolt alternative 🌟 Star if you like it! | first_party_docscensus | Source ↗ |
| — | LAION-AI/Open-Assistantedit_learning | OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so. | observed_behaviorcensus | Source ↗ |
| — | argilla-io/argillaedit_learning | Argilla is a collaboration tool for AI engineers and domain experts to build high-quality datasets | observed_behaviorcensus | Source ↗ |
| — | HumanSignal/label-studioedit_learning | Label Studio is a multi-type data labeling and annotation tool with standardized output format | observed_behaviorcensus | Source ↗ |
| 8 | vivekjoshy/openskill.pybayesian_preference | Multiplayer Rating System. No Friction. | first_party_docstop10 | Source ↗ |
| — | splitio/javascript-clientexperimentation | NodeJS and Browser SDK client for Split Software | observed_behaviorcensus | Source ↗ |
| — | ecosyste-ms/packagesasset_scoring | An open API service providing package, version and dependency metadata of many open source software ecosystems and registries. | first_party_docscensus | Source ↗ |
| — | Homebrew/homebrew-formula-analyticsregistry_ranking | 💀 Query Homebrew's analytics from the command-line (deprecated) | observed_behaviorcensus | Source ↗ |
100 mined records. Search and inspect the full agent output; nothing is collapsed into a top-ten headline.
| Rank | Identity | What the agent found | Evidence | Source |
|---|---|---|---|---|
| 1 | Min-gated asset ranking, never weighted average— | Asset rank is gated on the minimum evidence tier across families; a single weak family caps the asset regardless of other strengths | first_party_docstop10 | — |
| 2 | Rank by observed adaptation cost, not popularity— | Primary ranking signal is measured adaptation hours by category (branding, onboarding, settings, identity, navigation, data, ops) | first_party_docstop10 | — |
| 3 | Sparse-regime estimator selection— | Select estimators explicitly validated for tens of observations — Bayesian/Bradley-Terry over frequentist A/B | observed_behaviortop10 | — |
| 4 | Cold-start priors from industry archetype— | Prior initialised from the archetype and industry, updated as evidence arrives | first_party_docstop10 | — |
| 5 | Falsified-belief records as a first-class output— | Each run emits records of assumptions the outcome falsified, feeding the assumption ledger | first_party_docstop10 | — |
| — | Evidence-class-weighted signals— | Signals carry evidence class; only observed outcomes move rank materially | first_party_docsregister | — |
| — | Separate capability rank from archetype rank— | Score capability, binding and archetype separately | inferenceregister | — |
| — | Rank is scope-bound— | Ranks bind to industry and host scope | first_party_docsregister | — |
| 6 | Explicit exploration budget— | Declared fraction of builds may use a non-top-ranked candidate under review | observed_behaviortop10 | — |
| 7 | Exploration only within a safety envelope— | Exploration restricted to candidates passing all zero-tolerance gates; never on regulated workloads | first_party_docstop10 | — |
| — | Thompson-sampling-style selection over feasible sets— | Sample from the posterior over feasible candidates | observed_behaviorregister | — |
| — | Decay old evidence— | Time-decay weighting with declared half-life | inferenceregister | — |
| — | Demote on incident, not on complaint— | Only structured incident records affect rank | inferenceregister | — |
| — | Revocation propagates to rank instantly— | Revocation forces rank to floor and reopens dependent assemblies | first_party_docsregister | — |
| — | Rank explanations are mandatory— | Every rank carries its contributing signals | inferenceregister | — |
| — | Human override with recorded reason— | Override permitted, logged, and itself a learning signal | inferenceregister | — |
| — | Never rank on corpus size— | Explicitly excluded signal | first_party_docsregister | — |
| — | Reuse-shape-specific scoring— | Separate score dimensions per packaging mode | first_party_docsregister | — |
| — | Two-sided rank: fit and burden— | Score capability fit and maintenance burden separately, both visible | inferenceregister | — |
| — | Rank floor for unproven assets— | Assets without production evidence cannot exceed a declared ceiling | first_party_docsregister | — |
| — | Accepted plan as the primary positive signal— | Record the accepted AssemblyPlan with its capability set | first_party_docsregister | — |
| 8 | Client edit volume as a fit signal— | Edit counts by verb per capability feed rank | inferencetop10 | — |
| — | Rejected offers logged— | Log declined candidates | inferenceregister | — |
| — | Refusal log as demand signal— | Aggregate refusals into roadmap priorities | inferenceregister | — |
| — | Escape-hatch rate per capability— | Fork rate is a bounded-editing health metric | inferenceregister | — |
| — | Adaptation hours by category— | Record hours per adaptation category | first_party_docsregister | — |
| — | Glue size per binding— | Measure glue LOC and complexity per port binding | first_party_docsregister | — |
| — | Failed assumptions per build— | Structured record per falsified assumption | first_party_docsregister | — |
| — | Workflow defect count post-release— | Count defects in the accepted workflow after release | first_party_docsregister | — |
| — | Incident attributed per capability— | Requires the stable capability identifier | inferenceregister | — |
| — | Maintenance events per capability— | Log upgrade and repair events per capability | inferenceregister | — |
| — | Upgrade-break events— | Record when an upgrade broke client edits | observed_behaviorregister | — |
| — | Time-to-first-working-assembly— | Measure spec-confirmed to approved preview | first_party_docsregister | — |
| — | Reuse ratio per build— | Ports satisfied by admitted assets over total ports | first_party_docsregister | — |
| — | Human-touch minutes per accepted assembly— | Reviewer minutes recorded | first_party_docsregister | — |
| — | Escalation rate by trigger— | Count by named trigger | first_party_docsregister | — |
| — | Model tokens per accepted assembly— | Cost per accepted outcome | first_party_docsregister | — |
| 9 | UNDERDETERMINED results as contract defects— | Each names a missing contract field; aggregate drives contract evolution | first_party_docstop10 | — |
| — | Zero-tolerance events never averaged away— | Recorded separately; never contribute to an aggregate score | first_party_docsregister | — |
| — | Signal provenance recorded— | Every signal carries source, date and class | first_party_docsregister | — |
| — | Negative results retained— | Failed builds retained with reasons | first_party_docsregister | — |
| — | Client outcome signal, not just technical success— | Requires a client-agreed outcome rubric — currently absent | first_party_docsregister | — |
| 10 | Cross-client aggregation is opt-in and contractual— | Aggregation requires explicit contractual permission per client | first_party_docstop10 | — |
| — | Aggregate only structural facts, never content— | Learn from capability IDs, edit verbs, timings and outcomes — never client content, schema names or copy | inferenceregister | — |
| — | k-anonymity threshold before any aggregate— | Aggregates suppressed below a declared client count | observed_behaviorregister | — |
| — | Differential-privacy noise on published aggregates— | Calibrated noise with a declared budget | observed_behaviorregister | — |
| — | Per-tenant learning isolated by default— | Tenant-local learning unless aggregation is authorised | first_party_docsregister | — |
| — | No client identifiers in the learning store— | Pseudonymous cohort keys only | inferenceregister | — |
| — | Retention limit on learning signals— | Declared retention and deletion path | first_party_docsregister | — |
| — | Client deletion propagates to learning— | Signals removable and aggregates recomputable | inferenceregister | — |
| — | Learning store is read-only to composition agents— | Write path restricted to the deterministic pipeline | first_party_docsregister | — |
| — | Poisoning resistance— | Per-client influence caps | inferenceregister | — |
| — | No client-identifiable data in prompts— | Prompt packets carry structural facts only | first_party_docsregister | — |
| — | Audit log of learning updates— | Immutable log of rank changes and causes | inferenceregister | — |
| — | Regulated-industry exclusion— | Excluded from cross-client aggregation by default | first_party_docsregister | — |
| — | Published methodology— | Ranking methodology documented and versioned | observed_behaviorregister | — |
| — | Refresh cadence without assuming the large corpus— | Design refresh against the assets actually in hand; treat the corpus as an optional feed | first_party_docsregister | — |
| — | Refresh triggered by events, not only schedule— | Upstream release, advisory or licence change triggers refresh | inferenceregister | — |
| — | Source-bearing vs bundle-only distinction preserved— | Refresh tracks the distinction explicitly | first_party_docsregister | — |
| — | Deduplicate before counting— | Dedupe key per asset | first_party_docsregister | — |
| — | Deprecation on abandonment— | Maintenance signals trigger deprecation review | first_party_docsregister | — |
| — | Licence re-check on refresh— | Rights re-verified per refreshed revision | first_party_docsregister | — |
| — | Successor naming on deprecation— | Deprecation names a successor or an explicit none | first_party_docsregister | — |
| — | New-asset probation— | Probation period with a rank ceiling | inferenceregister | — |
| — | Taxonomy drift detection— | Periodic reconciliation with a versioned taxonomy | first_party_docsregister | — |
| — | Coverage gaps become research priorities— | EMPTY port results aggregate into a discovery backlog | first_party_docsregister | — |
| — | Stop rules on breadth— | Explicit breadth/depth stop rules | first_party_docsregister | — |
| — | Refresh receipts— | Dated receipt per refresh cycle | inferenceregister | — |
| — | Corpus provenance retained— | Immutable provenance chain | first_party_docsregister | — |
| — | Manual curation lane preserved— | Named human curation decisions recorded as signals | first_party_docsregister | — |
| — | Recipes version independently of assets— | Versioned recipes with pinned asset ranges | inferenceregister | — |
| — | Recipe success rate tracked— | Track accepted-build rate per recipe | inferenceregister | — |
| — | Successful compositions promote to recipes— | Repeated successful sets become candidate recipes | first_party_docsregister | — |
| — | Recurring edits promote to defaults— | Recurring edits become capability defaults or variants | inferenceregister | — |
| — | Archetype fit measured by workflow comprehension— | Test archetypes against task success | first_party_docsregister | — |
| — | Industry deltas recorded separately from archetypes— | Delta layer per industry | first_party_docsregister | — |
| — | Recipe deprecation on repeated failure— | Failure threshold triggers review | inferenceregister | — |
| — | Contract evolution driven by UNDERDETERMINED aggregates— | Missing-field counts prioritise contract work | first_party_docsregister | — |
| — | Binding patterns learned per port pair— | Successful bindings become reusable patterns | inferenceregister | — |
| — | Anti-pattern registry— | Record combinations that repeatedly failed | inferenceregister | — |
| — | Recipe cost model— | Predict cost from historical builds | inferenceregister | — |
| — | Recipes carry their own falsifiers— | Each recipe states what would retire it | first_party_docsregister | — |
| — | Cross-industry pattern detection— | Detect atoms recurring across industries | first_party_docsregister | — |
| — | Recipe provenance to first successful build— | Link recipe to the build that proved it | inferenceregister | — |
| — | REJECTED: ranking on stars/downloads— | Observed adaptation cost and outcomes | first_party_docsregister | — |
| — | REJECTED: corpus size as supply proof— | Qualification funnel metrics | first_party_docsregister | — |
| — | REJECTED: weighted-average asset scores— | Min-gated ranking | first_party_docsregister | — |
| — | REJECTED: cross-client learning by default— | Opt-in contractual aggregation | first_party_docsregister | — |
| — | REJECTED: learning from client content— | Structural facts only | inferenceregister | — |
| — | REJECTED: frequentist A/B on tens of builds— | Bayesian/sparse estimators | observed_behaviorregister | — |
| — | REJECTED: assuming the 1.3M corpus feed exists— | Design without it | first_party_docsregister | — |
| — | REJECTED: automatic promotion on green metrics— | Human admission decision retained | first_party_docsregister | — |
| — | REJECTED: model-authored ranking changes— | Deterministic pipeline owns writes | first_party_docsregister | — |
| — | REJECTED: treating vendor claims as loop evidence— | documented_methodology class required | observed_behaviorregister | — |
| — | REJECTED: single global rank— | Scope-bound ranks | first_party_docsregister | — |
| — | REJECTED: silent truncation of candidate sets— | Record candidates_dropped | first_party_docsregister | — |
| — | REJECTED: retiring assets on age alone— | Maintenance and outcome signals | inferenceregister | — |
| — | REJECTED: one learning model across all reuse shapes— | Shape-specific scoring | first_party_docsregister | — |
| — | REJECTED: contextual-bandit SaaS dependency— | Own the mechanism; treat vendors as replaceable | secondaryregister | — |
| — | REJECTED: claiming a learning loop exists today— | Design for cold start; state absence plainly | first_party_docsregister | — |
26 mined records. Search and inspect the full agent output; nothing is collapsed into a top-ten headline.
| Rank | Identity | What the agent found | Evidence | Source |
|---|---|---|---|---|
| — | P15-S-001local_artifact | lane contract | first_party_docsread_by_lane_owner | research/workstreams/2026-08-27-sprint-1/dispatch/S1-L5-editor-runtime-learning.md |
| — | P15-S-002local_artifact | depth/boundary/checkpoint contract | first_party_docsread_by_lane_owner | research/workstreams/2026-08-27-sprint-1/SPRINT-1-PROGRAM.md |
| — | P15-S-003local_artifact | run identity, opus-only policy | first_party_docsread_by_lane_owner | research/workstreams/2026-08-27-sprint-1/sprint-state.json |
| — | P15-S-004local_artifact | project boundary | first_party_docsread_by_lane_owner | AGENTS.md |
| — | P15-S-005local_artifact | current status | first_party_docsread_by_lane_owner | CURRENT_STATE.md |
| — | P15-S-006local_artifact | spine entry | first_party_docsread_by_lane_owner | knowledge/README.md |
| — | P15-S-007local_artifact | reuse shapes, plan-then-fill, unknowns | first_party_docsread_by_lane_owner | knowledge/00-MASTER-SYNTHESIS.md |
| — | P15-S-008local_artifact | A05/A07/A11/A15/A19/A20/A21/A22/A26/A37 | first_party_docsread_by_lane_owner | knowledge/02-ASSUMPTION-LEDGER.md |
| — | P15-S-009local_artifact | evidence routes | first_party_docsread_by_lane_owner | knowledge/03-EVIDENCE-MAP.md |
| — | P15-S-010local_artifact | P13/P14/P15 part records | first_party_docsread_by_lane_owner | site/system-map/data/parts.json |
| — | P15-S-011local_artifact | S1/S2 lane expectations | first_party_docsread_by_lane_owner | site/system-map/data/task-graph.json |
| — | P15-S-012local_artifact | picker/editing UX evidence | first_party_docsread_by_lane_owner | research/ui-pack-gallery-ux-2026-08-27.md |
| — | P15-S-013local_artifact | knob vector, integration modes, preference learning | first_party_docsread_by_lane_owner | research/actionmodel-builder-research-2026-08-26/phase-8/external-opus-inputs/CLAUDE-LANES-SYNTHESIS.md |
| — | P15-S-014local_artifact | lifecycle, nine families, T0-T4 anti-averaging, deferred status | first_party_docsread_by_lane_owner | research/actionmodel-builder-research-2026-08-26/phase-8/lanes/01-universal-block-framework/outputs/universal-block-framework.md |
| — | P15-S-015local_artifact | staged writes, authority ladder, ABSTAIN, kill rules | first_party_docsread_by_lane_owner | research/actionmodel-builder-research-2026-08-26/phase-8/lanes/05-composition-agent-evals/outputs/composition-agent-architecture.md |
| — | P15-S-016local_artifact | runtime decomposition, build-run state machine, sandbox/edge decisions | first_party_docsread_by_lane_owner | architecture/feature-matrix/ARCHITECTURE.md |
| — | P15-S-017local_artifact | seven gates, G5 receipts, N01-N15 | first_party_docsread_by_lane_owner | research/actionmodel-builder-research-2026-08-26/phase-2/outputs/evaluation-and-admission-plan.md |
| — | P15-S-018local_artifact | receipt schemas, kill/hold | first_party_docsread_by_lane_owner | research/actionmodel-builder-research-2026-08-26/phase-4/outputs/pilot-receipt-runbook.md |
| — | P15-S-019local_artifact | claim classes, stop rules | first_party_docsread_by_lane_owner | research/actionmodel-builder-research-2026-08-26/phase-3/outputs/decision-ledger.md |
| — | P15-S-020local_artifact | 67-surface census | first_party_docsread_by_lane_owner | research/actionmodel-builder-research-2026-08-26/phase-2/outputs/platform-deepdives.md |
| — | P15-S-021local_artifact | commercial matrix, coverage ledger | first_party_docsread_by_lane_owner | research/packs/source-files/builder-ecosystem-report.md |
| — | P15-S-022local_artifact | factory loop, failed-assumptions output | first_party_docsread_by_lane_owner | [private local source withheld] |
| — | P15-S-023local_artifact | operated mixed-shape runtime, two-object rollback, required proof | first_party_docsread_by_lane_owner | [private local source withheld] |
| — | P15-S-024run_artifact | commercial census produced by this run; 22 records | observed_behaviorproduced_by_lane | research/workstreams/p15-learning-feedback/runs/2026-08-27-sprint-1-fable/top-companies.jsonl |
| — | P15-S-025run_artifact | OSS census produced by this run; 97 records | observed_behaviorproduced_by_lane | research/workstreams/p15-learning-feedback/runs/2026-08-27-sprint-1-fable/top-repos.jsonl |
| — | P15-S-026run_artifact | innovation register produced by this run; 100 records | observed_behaviorproduced_by_lane | research/workstreams/p15-learning-feedback/runs/2026-08-27-sprint-1-fable/innovation-register.jsonl |
Source: site/system-map/data/parts.json#P15 · preserved research datasets remain inline below their source-backed synthesis.