Current synthesis from the research packet. Exact records and complete source artifacts remain visible below.
Design and evaluate the active-learning experiment, including stopping rules and a reproducible DesignDNA output.
knowledge/06-DESIGN-GRAMMAR-AND-PREFERENCE-PROTOCOL.mdresearch/ui-pick-to-spec-2026-08-27.mdresearch/token-pack-science-2026-08-27.mdhttps://actionist-taste.pages.dev/Lovable lets a user bring a design system and then enforces it, which assumes the client can author one. Learning taste from a short sequence of structured comparisons gets a defensible visual identity from a client who could not have written the brief.
113 mined records. Search and inspect the full agent output; nothing is collapsed into a top-ten headline.
| Rank | Identity | What the agent found | Evidence | Source |
|---|---|---|---|---|
| — | Netflix title pickermedia-cold-start | Multi-select grid of titles at profile creation; explicitly OPTIONAL — skipping falls back to a diverse popular set; later engagement 'supersedes' the initial picks, recent outweighing old. Count unstated; single screen, non-adaptive. | observedSTEAL — the documented supersession rule is the cleanest published answer to preference drift | Source ↗ |
| — | Netflix Taste Preferences surveymedia-cold-start | Separate always-available survey under Account > My Profile covering Moods, Genre, Release Date, Language; user-initiated, no stopping rule. | observedstudy — precedent for a re-entrant preference surface distinct from onboarding | Source ↗ |
| — | Spotify onboarding artist pickermedia-cold-start | Multi-select grid, minimum 3 artists, presented before first home screen; described as cyclic (selection then refinement). No documented outside option. | inferredstudy — the non-re-entrant design is a documented user complaint worth avoiding | Source ↗ |
| — | Apple Music bubble pickermedia-cold-start | Genre/artist bubble multi-select at onboarding, tap-to-grow weighting. | hypothesisdrop unless verified | Source ↗ |
| — | Pandora thumbsmedia-cold-start | Continuous binary thumb up/down on played tracks feeding station personalisation. | hypothesisreference only | Source ↗ |
| — | YouTube Music onboardingmedia-cold-start | Artist multi-select at first run. | hypothesisdrop unless verified | Source ↗ |
| — | Tidal onboardingmedia-cold-start | Artist/genre selection at signup. | hypothesisdrop unless verified | Source ↗ |
| — | Deezer Flow onboardingmedia-cold-start | Genre selection seeding Flow. | hypothesisdrop unless verified | Source ↗ |
| — | Last.fm scrobble-based tastemedia-cold-start | Purely revealed-preference (listening history), no elicitation step. | hypothesiscontrast — the pure revealed-preference pole | Source ↗ |
| — | Stitch Fix Style Shufflefashion | Binary thumbs up/down on garment images, launched Mar 2018, ~10B interactions, ~1M players/month (~25% of active clients). Ratings update the 'Latent Style' model in REAL TIME; feeds a Style Graph distilling 10B data points into 10M coordinates. Unbounded — no fixed N, user-terminated. Batched ~10 items, a few batches/day. | observedSTEAL — closest large-scale proof that binary image preference converges to a usable latent vector | Source ↗ |
| — | Stitch Fix 'candy vs medicine' item selectionfashion | Shown items are deliberately mixed: 'candy' (near-certain thumbs-up, engagement) and 'medicine' (items the model specifically needs information about). This is explicit exploration/exploitation balancing inside a consumer UI. | inferredSTEAL — the engagement/information tradeoff is exactly our re-roll-vs-information problem | Source ↗ |
| — | Stitch Fix onboarding Style Profilefashion | Long-form intake covering sizes, price bands, style preferences before first Fix. | hypothesisstudy if access obtained | Source ↗ |
| — | Threadfashion | Style quiz feeding a stylist+algorithm hybrid. | hypothesisdrop unless verified | Source ↗ |
| — | Wantable style quizfashion | Edit-based style quiz before curated box. | hypothesisdrop unless verified | Source ↗ |
| — | Nuulyfashion | Rental subscription with style preferences. | hypothesisdrop unless verified | Source ↗ |
| — | DailyLookfashion | Style quiz feeding stylist box. | hypothesisdrop unless verified | Source ↗ |
| — | Armoirefashion | Rental with algorithmic closet personalisation. | hypothesisdrop unless verified | Source ↗ |
| — | Rent the Runwayfashion | Size/fit and style preference capture. | hypothesisdrop unless verified | Source ↗ |
| — | Nordstrom Trunk Clubfashion | Stylist intake questionnaire. | hypothesisdrop | Source ↗ |
| — | Frank And Oak style planfashion | Style quiz feeding subscription box. | hypothesisdrop unless verified | Source ↗ |
| — | Pinterest interest pickerinterest-picker | Mandatory multi-select of >=5 topics from an image+keyword grid; single screen, non-adaptive; framed as 'Follow 5 topics' with an explicit promise of a custom home feed. Localising the topic set per country/gender produced a large activation lift and 5-10% higher return rate. | inferredSTEAL — the localisation result is the strongest evidence that STIMULUS CHOICE, not model, drives elicitation quality | Source ↗ |
| — | Pinterest user-interest clusters (UIC)interest-picker | Pinterest REPLACED onboarding followed-interests as a retrieval condition with user interest clusters derived from actual engagement, because the onboarding signal 'skewed heavily toward dominant interests' and was 'static, not evolving with behavior'. | inferredCRITICAL COUNTER-EVIDENCE — a major platform deprecated its stated-preference onboarding signal in favour of revealed preference | Source ↗ |
| — | Reddit onboarding topic pickerinterest-picker | Topic multi-select seeding the home feed. | hypothesisdrop unless verified | Source ↗ |
| — | TikTok interest selectinterest-picker | Category multi-select at signup; famously superseded fast by watch behaviour. | hypothesisstudy if verified — another stated-vs-revealed case | Source ↗ |
| — | Substack onboardinginterest-picker | Topic selection seeding recommendations. | hypothesisdrop unless verified | Source ↗ |
| — | Medium topic followinterest-picker | Topic multi-select at signup. | hypothesisdrop unless verified | Source ↗ |
| — | Flipboard topic pickerinterest-picker | Topic selection building a magazine feed. | hypothesisdrop unless verified | Source ↗ |
| — | Quora topic selectioninterest-picker | Topic multi-select at signup. | hypothesisdrop unless verified | Source ↗ |
| — | Havenly interior design style quizinteriors | Grid of ~48 room inspiration images, multi-select with NO cap ('Select the rooms that make you swoon' / 'Pick as many as you want'), and an explicit outside option: 'I don't like these. Skip.' Output is a style profile with a primary style plus sub-styles, handed to a matched designer. | observedSTEAL — the explicit 'I don't like these' outside option in a live consumer flow is direct precedent for our re-roll | Source ↗ |
| — | Wayfair design style quizinteriors | Choose favourite photo from FOUR options per question, tally letters to a named style (boho, modern, ...). Fixed sequence, self-scored, no adaptivity, no outside option. Results funnel to shoppable category links. | inferredSTUDY — a 4-up image forced choice at consumer scale, but non-adaptive and self-scored | Source ↗ |
| — | Jordan's Furniture Find Your Styleinteriors | 5-step flow, select up to 2 images per step, validation blocks progress without a selection (i.e. NO outside option). Output is a prose style description. | inferredstudy — the forced-selection validation is the anti-pattern our outside option avoids | Source ↗ |
| — | Modsyinteriors | Style quiz feeding 3D room renders. | hypothesisdrop — do not present as a live competitor | Source ↗ |
| — | CORT Furniture style quizinteriors | Quiz results lead to curated product lists per named style. | inferredreference only | Source ↗ |
| — | Furnitureland South style quizinteriors | Short instinct-based style quiz. | inferredreference only | Source ↗ |
| — | Waypoint Living Spaces cabinet quizinteriors | Hybrid text/image quiz over colour palettes, chair preference, patterns; emails results as lead capture. | inferredreference only | Source ↗ |
| — | IKEA room style toolsinteriors | Room planner and style inspiration surfaces. | hypothesisdrop unless verified | Source ↗ |
| — | Articleinteriors | NO public image-based style quiz found; personalisation is via catalogue browse and filters. | observedNEGATIVE RESULT — a major furniture retailer with no style quiz | Source ↗ |
| — | Joybirdinteriors | NO public image-based style quiz found; personalisation is via heavy per-product customisation (fabrics, cushion feel, legs) plus a free design service. | observedNEGATIVE RESULT — chooses configuration over elicitation | Source ↗ |
| — | Houzzinteriors | Photo save/ideabook behaviour as implicit style signal. | hypothesisstudy if verified | Source ↗ |
| — | Room & Boardinteriors | Design services intake. | hypothesisdrop unless verified | Source ↗ |
| — | Warby Parker frames quizphysical-goods | Short quiz ('a few quick questions'); a company promo described 5 questions. Output is frame recommendations by colour, shape, style, feeding Home Try-On (5 frames free). | inferredstudy — short-quiz-to-physical-trial is the commercial pattern, count unverified | Source ↗ |
| — | Glasses.com virtual try-onphysical-goods | Face-scan plus style preference capture. | hypothesisdrop unless verified | Source ↗ |
| — | Sherwin-Williams ColorSnapphysical-goods | Colour discovery and palette tools. | hypothesisdrop unless verified | Source ↗ |
| — | Behr colour toolsphysical-goods | Colour selection and palette tools. | hypothesisdrop | Source ↗ |
| — | Prose hair consultationbeauty | Consultation covering 85+ data points (hair type, porosity, wash frequency, zip code for humidity/water hardness/pollution, diet, stress). Fixed non-adaptive intake; output is a bespoke formula. | inferredCONTRAST — the maximalist pole: 85+ questions. Shows what we are choosing NOT to do | Source ↗ |
| — | Function of Beauty quizbeauty | Materially SHORTER quiz than Prose — skips age, location, hair length, shedding, wash frequency; focuses on hair type and goals plus fragrance/bottle colour. Reportedly drives 80% of business. | inferredstudy — the short-quiz counterpoint to Prose in the same category | Source ↗ |
| — | Curologybeauty | Skin intake plus photos feeding a prescriber. | hypothesisdrop unless verified | Source ↗ |
| — | Care/ofbeauty | Supplement quiz. | hypothesisdrop unless verified | Source ↗ |
| — | Atlas Coffee Club quizbeauty | Coffee taste quiz feeding subscription. | hypothesisdrop | Source ↗ |
| — | Firstleaf winebeauty | Taste quiz plus per-bottle rating feedback loop — a genuine ongoing preference learner. | hypothesisstudy if verified — the rating feedback loop is architecturally close to ours | Source ↗ |
| — | Winc wine quizbeauty | Palate profile quiz. | hypothesisdrop unless verified | Source ↗ |
| — | Hinge Most Compatibledating | NO explicit elicitation step. Preference rankings are INFERRED by ML from like/pass behaviour over a rolling 24h window, then Gale-Shapley produces a stable matching. Refreshes every 24h. | inferredSTUDY — the purest revealed-preference architecture in the denominator; the 24h rolling window is a concrete drift-handling design | Source ↗ |
| — | OkCupid question weightingdating | User answers questions AND states importance weight AND the acceptable answer from a partner — a rare three-part elicitation. | hypothesisstudy if verified — explicit importance weighting is directly relevant to per-knob weighting | Source ↗ |
| — | Tinder swipedating | Binary swipe as continuous implicit preference signal. | hypothesisreference only | Source ↗ |
| — | Coffee Meets Bageldating | Curated daily matches from stated plus revealed preference. | hypothesisdrop unless verified | Source ↗ |
| — | Midjourney Style Creatordesign-tooling | SHIPPED ADAPTIVE VISUAL PREFERENCE ELICITOR. User picks from a grid of style images; the tool uses 'the styles you pick (and the ones you don't!)' to build a --sref code. Each regenerated preview set is one refinement round. Documented convergence: early rounds vary a lot, 'Most styles stabilize after 5-10 rounds', rounds 10-15 add detail, 'Past round 15, changes are small and subtle'. Skipping is available but explicitly NON-informative: 'skipping does not affect your style development'. Output is a reusable style code. User-terminated via End Session. | observedSTEAL — the single closest shipped analogue to P06. Note we should DIVERGE on one point: they make skipping non-informative; we should treat the outside option as weak negative evidence | Source ↗ |
| — | Midjourney Style Tuner (/tune, legacy)design-tooling | Two modes: PAIRWISE (two styles at a time, click the preferred one, leave the middle box selected to SKIP the pair) or grid 'pick favourites'. 32-pair generation typical. Even a single pick generates a style; fewer picks produce a bolder style. | inferredSTEAL — an explicit three-way pairwise-with-neutral-middle control is exactly the outside-option affordance we need, and it shipped | Source ↗ |
| — | Figma First Draft style controlsdesign-tooling | NOT elicitation — DIRECT MANIPULATION. Theme controls expose light/dark toggle, preset colours or colour wheel, a border-radius SLIDER, a spacing SLIDER, and title/body/label font selection. No comparisons, no learning, no stopping rule. | observedCONTRAST — the closest design-tool competitor solves the same problem with sliders instead of preference learning. Our differentiator is that a client does not know their radius in pixels | Source ↗ |
| — | Looka logo maker style quizdesign-tooling | Fixed sequence: business name + industry, then select logo STYLES from visual examples, then up to THREE colours, then icon concepts. Non-adaptive: 'the AI works within a structured decision tree'. Constrains the solution space BEFORE generation. Under five minutes. | inferredSTUDY — closest commercial 'visual picks constrain a generator' flow, but non-adaptive and explicitly criticised for having no box for unusual blends | Source ↗ |
| — | Canva Brand Kit + on-brand generationdesign-tooling | NOT elicitation. A stored brand layer (logos, colours, fonts, templates) is APPLIED to designs; Canva AI can extract brand context from a public website URL. Brand Controls can restrict a design to only Brand Kit colours and fonts. | observedSTEAL the CONSTRAINT model — 'Brand Controls' restricting a design to only kit values is exactly our closed-world token guarantee, shipped | Source ↗ |
| — | Base44 workspace design systemsdesign-tooling | A default design system is preselected before a new app's first prompt; users can choose another via chat. Stores brand basics, colours, fonts, logo, radius, shadows, spacing, components. | inferredSTUDY — closest AI-builder precedent for pre-build design-system selection, but selection not elicitation | Source ↗ |
| — | Wix ADI style stepdesign-tooling | HISTORICAL. ADI's fixed style-question flow was RETIRED 10 Nov 2024 and is unavailable for new sites; the current Wix builder is prompt-based ('Describe your vision'). | observedNEGATIVE RESULT — must not be cited as a live competitor; a style-quiz onboarding was retired in favour of prompting | Source ↗ |
| — | v0design-tooling | React/Tailwind/shadcn generation with reusable style presets and design-system instructions; no closed pre-build token-pack gallery confirmed. | inferredcontrast | Source ↗ |
| — | Lovabledesign-tooling | Screenshot/Figma import and visual edits documented; no pre-build finite preset gallery confirmed. | inferredcontrast | Source ↗ |
| — | Boltdesign-tooling | Prompt generation, imports, deployment; no finite pre-build theme picker confirmed. | inferredcontrast | Source ↗ |
| — | Replit Agent / Design Modedesign-tooling | Visual Editor/Design Mode and Figma import; cited custom-theme docs concern the editor's own appearance, not generated-app tokens. | inferredcontrast | Source ↗ |
| — | Uizarddesign-tooling | Theme generation and style extraction from screenshots. | hypothesisdrop unless verified | Source ↗ |
| — | Galileo AIdesign-tooling | Prompt-to-UI generation. | hypothesisdrop unless verified | Source ↗ |
| — | Relumedesign-tooling | Sitemap-to-wireframe generation with style application. | hypothesisdrop unless verified | Source ↗ |
| — | Durabledesign-tooling | AI website builder with rapid theme regeneration. | hypothesisdrop unless verified | Source ↗ |
| — | Hostinger AI builderdesign-tooling | Prompt-based site generation with theme options. | hypothesisdrop unless verified | Source ↗ |
| — | Tailor Brandsdesign-tooling | Logo/brand quiz with style selection. | hypothesisdrop unless verified | Source ↗ |
| — | Brandmarkdesign-tooling | Keyword plus colour/style selection generating logo sets. | hypothesisdrop unless verified | Source ↗ |
| — | Squarespace onboardingdesign-tooling | Goal and category selection preceding template recommendation. | hypothesisdrop unless verified | Source ↗ |
| — | Framer marketplacedesign-tooling | Curated template browse and customise; template not token pack. | inferredreference | Source ↗ |
| — | Webflow templatesdesign-tooling | Browse/search/preview; 'Preview in Designer' allows trying edits without saving; selecting creates a new project. | inferredSTEAL the PREVIEW-BEFORE-COMMIT pattern — reduces commitment anxiety at the pick moment | Source ↗ |
| — | Wix Templatesdesign-tooling | Card to View/Edit to customise; very clear non-technical picker UX. | inferredreference | Source ↗ |
| — | ThemeForest / Envatodesign-tooling | Faceted search, sales/rating metadata, separate live preview. | inferredreference | Source ↗ |
| — | tweakcndesign-tooling | Live visual editing of shadcn/Tailwind themes with code export; direct manipulation, not elicitation. | inferredcontrast | Source ↗ |
| — | Sawtooth MaxDiffmeasurement-vendor | THE HARDEST SIZING RULE IN THE DENOMINATOR. 'we recommend displaying either four or five items at a time (per set or question)'; never more than half the study's item count per set; ask enough sets 'such that each item has the opportunity to appear from three to five times per respondent'; formula '3K/k' where K = total items and k = items per set. Beyond ~5 items per set 'The gains in precision of the estimates are minimal.' Assumes HB estimation. Sparse MaxDiff relaxes this to population coverage (each item seen 500-1000 times across the population). | observedADOPT-METHOD — 3K/k is the most defensible published question-count formula available, and the '4-5 per screen' guidance independently corroborates our 4-up recommendation | Source ↗ |
| — | Sawtooth ACBCmeasurement-vendor | Genuinely ADAPTIVE three-phase design: BYO configuration, then a screening phase, then choice tasks. Screening presents 3-5 concepts per page and periodically asks whether a level is a 'Must-Have'; once a cutoff is set, all further concepts satisfy it. Also captures 'Unacceptables'. Length ~7-15 min, 2-3x a standard CBC. Output is part-worth utilities PLUS explicit non-compensatory rules. | observedSTEAL the MUST-HAVE / UNACCEPTABLE mechanism — a hard constraint that permanently prunes the stimulus space is exactly how brand constraints should enter our elicitation | Source ↗ |
| — | Sawtooth CBCmeasurement-vendor | Choice-based conjoint: typically four concepts per task, orthogonal/balanced design, HB utilities per respondent. Non-adaptive. | inferredreference — the non-adaptive baseline our adaptive design must beat | Source ↗ |
| — | Conjointlymeasurement-vendor | Conjoint and MaxDiff survey platform. | hypothesisdrop unless verified | Source ↗ |
| — | Qualtrics Conjointmeasurement-vendor | Choice-based conjoint capped at <10k responses. | inferredreference | Source ↗ |
| — | 1000minds PAPRIKAmeasurement-vendor | PAPRIKA method: adaptive pairwise ranking of partial profiles, eliminating implied pairs by transitivity — algorithmically the closest commercial method to our adaptive design. | hypothesisHIGH PRIORITY TO VERIFY — transitivity-based pair elimination is directly applicable | Source ↗ |
| — | Displayrmeasurement-vendor | MaxDiff and conjoint analysis platform. | hypothesisdrop unless verified | Source ↗ |
| — | Q Research Softwaremeasurement-vendor | MaxDiff experimental design tooling. | inferredreference | Source ↗ |
| — | Alchemermeasurement-vendor | Survey platform with conjoint support. | hypothesisdrop unless verified | Source ↗ |
| — | QuestionPro conjointmeasurement-vendor | Conjoint and MaxDiff modules. | hypothesisdrop unless verified | Source ↗ |
| — | SKIMmeasurement-vendor | Choice-modelling consultancy. | hypothesisdrop unless verified | Source ↗ |
| — | Kantarmeasurement-vendor | Market research including conjoint. | hypothesisdrop unless verified | Source ↗ |
| — | Protobrandmeasurement-vendor | Metaphor-elicitation and implicit association research. | hypothesisstudy if verified — implicit/projective elicitation is a genuinely different family | Source ↗ |
| — | Optimizely contextual banditsexperimentation | PER-USER, not population: 'contextual bandits pick a winning variation for each user based on their contextual profile'. Model starts at 100% exploration then exploits. NO sticky bucketing — each visitor gets the best variation for them at that moment. No statistical significance is calculated. User attributes cannot be added or removed once started. | inferredSTUDY — the no-sticky-bucketing and no-significance properties are the honest cost of per-user personalisation, and we inherit both | Source ↗ |
| — | Statsig Autotune AIexperimentation | Contextual bandits positioned between A/B tests and full ML; hourly model retraining. Explicit caveat that bandits 'excel at choosing between known options based on context but won't generate new content'. | inferredstudy — 'choosing between known options' is precisely our closed-pack framing | Source ↗ |
| — | Spotify separate personalisation/experimentation stacksexperimentation | Argues bandits and experiments are NOT substitutes: with a contextual bandit there is no single 'best button', only a 'best system', so you still need an experiment comparing it to the static baseline. The bandit is a feature you built, not an experimental method. | inferredCRITICAL METHOD NOTE — our preference learner must itself be A/B tested against a static default pack, or we cannot claim it works | Source ↗ |
| — | Dynamic Yieldexperimentation | Multi-armed and contextual bandit optimisation for commerce personalisation. | inferredreference | Source ↗ |
| — | Kameleoonexperimentation | Contextual bandit personalisation. | inferredreference | Source ↗ |
| — | VWOexperimentation | A/B and personalisation platform. | hypothesisdrop unless verified | Source ↗ |
| — | GrowthBookexperimentation | Open-source feature flagging and experimentation. | hypothesisdrop unless verified | Source ↗ |
| — | Amplitude Experimentexperimentation | Experimentation with adaptive allocation. | hypothesisdrop unless verified | Source ↗ |
| — | LaunchDarklyexperimentation | Feature management with experimentation. | hypothesisdrop unless verified | Source ↗ |
| — | Monetateexperimentation | Commerce personalisation platform. | hypothesisdrop unless verified | Source ↗ |
| — | Actionist taste picker demolocal-prior-art | OUR OWN EXISTING PROTOTYPE. Four rendered interface cards per round ('every card is a bundle of five knobs — your pick is evidence about each one'), with an explicit outside option labelled 'none of these' plus 'start over'. ~10 picks total; later rounds target the 'least-resolved knob', holding other inferred preferences steady. Five knobs varied (palette, radius, type, density, shadow) of a seven-knob schema. Live belief panel headed 'What the system believes about you'. Statistics are win-rate counting with Laplace smoothing — a zeroth-order Bradley-Terry; the page itself names choix or a GP preference model as the production intent. | observedADOPT the interaction model — it already implements the recommended format, outside option and least-resolved-knob targeting; replace the estimator and add a real stopping rule | Source ↗ |
| 1 | Midjourney Style Creatordesign-tooling | SHIPPED ADAPTIVE VISUAL PREFERENCE ELICITOR. User picks from a grid of style images; the tool uses 'the styles you pick (and the ones you don't!)' to build a --sref code. Each regenerated preview set is one refinement round. Documented convergence: early rounds vary a lot, 'Most styles stabilize after 5-10 rounds', rounds 10-15 add detail, 'Past round 15, changes are small and subtle'. Skipping is available but explicitly NON-informative: 'skipping does not affect your style development'. Output is a reusable style code. User-terminated via End Session. | observedSTEAL — the single closest shipped analogue to P06. Note we should DIVERGE on one point: they make skipping non-informative; we should treat the outside option as weak negative evidence | Source ↗ |
| 2 | Sawtooth MaxDiffmeasurement-vendor | THE HARDEST SIZING RULE IN THE DENOMINATOR. 'we recommend displaying either four or five items at a time (per set or question)'; never more than half the study's item count per set; ask enough sets 'such that each item has the opportunity to appear from three to five times per respondent'; formula '3K/k' where K = total items and k = items per set. Beyond ~5 items per set 'The gains in precision of the estimates are minimal.' Assumes HB estimation. Sparse MaxDiff relaxes this to population coverage (each item seen 500-1000 times across the population). | observedADOPT-METHOD — 3K/k is the most defensible published question-count formula available, and the '4-5 per screen' guidance independently corroborates our 4-up recommendation | Source ↗ |
| 3 | Stitch Fix Style Shufflefashion | Binary thumbs up/down on garment images, launched Mar 2018, ~10B interactions, ~1M players/month (~25% of active clients). Ratings update the 'Latent Style' model in REAL TIME; feeds a Style Graph distilling 10B data points into 10M coordinates. Unbounded — no fixed N, user-terminated. Batched ~10 items, a few batches/day. | observedSTEAL — closest large-scale proof that binary image preference converges to a usable latent vector | Source ↗ |
| 4 | Sawtooth ACBCmeasurement-vendor | Genuinely ADAPTIVE three-phase design: BYO configuration, then a screening phase, then choice tasks. Screening presents 3-5 concepts per page and periodically asks whether a level is a 'Must-Have'; once a cutoff is set, all further concepts satisfy it. Also captures 'Unacceptables'. Length ~7-15 min, 2-3x a standard CBC. Output is part-worth utilities PLUS explicit non-compensatory rules. | observedSTEAL the MUST-HAVE / UNACCEPTABLE mechanism — a hard constraint that permanently prunes the stimulus space is exactly how brand constraints should enter our elicitation | Source ↗ |
| 5 | Havenly interior design style quizinteriors | Grid of ~48 room inspiration images, multi-select with NO cap ('Select the rooms that make you swoon' / 'Pick as many as you want'), and an explicit outside option: 'I don't like these. Skip.' Output is a style profile with a primary style plus sub-styles, handed to a matched designer. | observedSTEAL — the explicit 'I don't like these' outside option in a live consumer flow is direct precedent for our re-roll | Source ↗ |
| 6 | Pinterest user-interest clusters (UIC)interest-picker | Pinterest REPLACED onboarding followed-interests as a retrieval condition with user interest clusters derived from actual engagement, because the onboarding signal 'skewed heavily toward dominant interests' and was 'static, not evolving with behavior'. | inferredCRITICAL COUNTER-EVIDENCE — a major platform deprecated its stated-preference onboarding signal in favour of revealed preference | Source ↗ |
| 7 | Actionist taste picker demolocal-prior-art | OUR OWN EXISTING PROTOTYPE. Four rendered interface cards per round ('every card is a bundle of five knobs — your pick is evidence about each one'), with an explicit outside option labelled 'none of these' plus 'start over'. ~10 picks total; later rounds target the 'least-resolved knob', holding other inferred preferences steady. Five knobs varied (palette, radius, type, density, shadow) of a seven-knob schema. Live belief panel headed 'What the system believes about you'. Statistics are win-rate counting with Laplace smoothing — a zeroth-order Bradley-Terry; the page itself names choix or a GP preference model as the production intent. | observedADOPT the interaction model — it already implements the recommended format, outside option and least-resolved-knob targeting; replace the estimator and add a real stopping rule | Source ↗ |
| 8 | Figma First Draft style controlsdesign-tooling | NOT elicitation — DIRECT MANIPULATION. Theme controls expose light/dark toggle, preset colours or colour wheel, a border-radius SLIDER, a spacing SLIDER, and title/body/label font selection. No comparisons, no learning, no stopping rule. | observedCONTRAST — the closest design-tool competitor solves the same problem with sliders instead of preference learning. Our differentiator is that a client does not know their radius in pixels | Source ↗ |
| 9 | Spotify separate personalisation/experimentation stacksexperimentation | Argues bandits and experiments are NOT substitutes: with a contextual bandit there is no single 'best button', only a 'best system', so you still need an experiment comparing it to the static baseline. The bandit is a feature you built, not an experimental method. | inferredCRITICAL METHOD NOTE — our preference learner must itself be A/B tested against a static default pack, or we cannot claim it works | Source ↗ |
| 10 | Canva Brand Kit + on-brand generationdesign-tooling | NOT elicitation. A stored brand layer (logos, colours, fonts, templates) is APPLIED to designs; Canva AI can extract brand context from a public website URL. Brand Controls can restrict a design to only Brand Kit colours and fonts. | observedSTEAL the CONSTRAINT model — 'Brand Controls' restricting a design to only kit values is exactly our closed-world token guarantee, shipped | Source ↗ |
119 mined records. Search and inspect the full agent output; nothing is collapsed into a top-ten headline.
| Rank | Identity | What the agent found | Evidence | Source |
|---|---|---|---|---|
| — | lucasmaystre/choixrepo | Inference for Luce-choice-axiom models: Bradley-Terry, Plackett-Luce, Thurstone; MM, Lu-Boutilier, opt & EP methods on pairwise/top-1/ranking data. | observedadopt — the single best fit for fitting BT over 7 knobs from pairwise data in Python | Source ↗ |
| — | sublee/trueskillrepo | Python TrueSkill: Bayesian factor-graph skill rating, Gaussian belief per player updated by match outcomes. | observedREJECT for paid client work — use openskill.py instead | Source ↗ |
| — | vivekjoshy/openskill.pyrepo | Clean-room multiplayer Bayesian rating (Weng-Lin models: BradleyTerryFull/Part, PlackettLuce, ThurstoneMostellerFull/Part). No Microsoft trademark encumbrance. | observedadopt — the licence-safe TrueSkill substitute | Source ↗ |
| — | scttcper/ts-trueskillrepo | TypeScript port of the Python TrueSkill package — usable browser-side. | observedstudy — only if elicitation must run client-side in TS | Source ↗ |
| — | hturner/BradleyTerry2repo | R: Bradley-Terry models incl. structured versions where abilities are a linear predictor of item covariates — the 'knob' formulation we actually need. | observedstudy — the covariate-structured BT is the right model form; reimplement rather than link | Source ↗ |
| — | EllaKaye/BradleyTerryScalablerepo | R package fitting BT to large sparse comparison datasets. | observedreject on licence grounds | Source ↗ |
| — | hturner/PlackettLucerepo | R: Plackett-Luce models for rankings, handles ties and partial rankings. | observedstudy — matters if we choose N-way over pairwise | Source ↗ |
| — | cran/prefmodrepo | R: fits paired-comparison preference models (Bradley-Terry loglinear), handles undecided/ties and subject covariates. | observedstudy | Source ↗ |
| — | UDST/choicemodelsrepo | Python discrete choice modelling (MNL, large-alternative sampling) from the UrbanSim group. | observedstudy | Source ↗ |
| — | timothyb0912/pylogitrepo | Python conditional logit / MNL estimation for discrete choice experiments. | observedstudy | Source ↗ |
| — | arteagac/xlogitrepo | GPU-accelerated mixed logit estimation. | observedreject — wrong scale, restrictive licence | Source ↗ |
| — | michelbierlaire/biogemerepo | Mature maximum-likelihood estimation of parametric discrete choice models. | observedstudy with licence caution | Source ↗ |
| — | gsbDBI/torch-choicerepo | PyTorch choice modelling (logit, nested logit) — GPU, differentiable, composes with learned embeddings. | observedadopt-candidate if the taste model must sit inside a torch pipeline | Source ↗ |
| — | meta-pytorch/botorchrepo | Bayesian optimization in PyTorch. Ships PairwiseGP, PairwiseLaplaceMarginalLogLikelihood, and qEUBO — a production-grade preferential-BO stack. | observedADOPT — the single most load-bearing repo for this lane | Source ↗ |
| — | facebook/Axrepo | Adaptive experimentation platform over BoTorch; service API manages the ask/tell loop and trial state. | observedadopt — use for the ask/tell loop rather than hand-rolling | Source ↗ |
| — | facebookresearch/qEUBOrepo | Reproducible code for the AISTATS 2023 qEUBO paper — decision-theoretic acquisition function for preferential BO. | observedstudy — read it, then use the BoTorch-integrated version | Source ↗ |
| — | AaltoPML/PPBOrepo | Projective Preferential Bayesian Optimization — queries along projections, designed for human-in-the-loop high-dim preference elicitation. | observedstudy — projection idea is directly relevant to 7 knobs | Source ↗ |
| — | CyberAgentAILab/preferentialBOrepo | ICML 2023 'Towards Practical Preferential Bayesian Optimization' implementation. | observedstudy | Source ↗ |
| — | ja-thomas/pbmohporepo | Preferential Bayesian multi-objective hyperparameter optimization. | observeddrop | Source ↗ |
| — | yuki-koyama/sequential-line-searchrepo | C++/Python preferential BO library implementing SIGGRAPH 2017 sequential line search — the user picks a point on a 1D slider through a high-dim design space. | observedADOPT/STEAL — closest published prior art to our exact problem | Source ↗ |
| — | yuki-koyama/sequential-galleryrepo | SIGGRAPH 2020 sequential-plane-search + adaptive grid gallery UI for visual design optimization. | observedADOPT/STEAL — the gallery interaction model is the design we want | Source ↗ |
| — | modAL-python/modALrepo | Modular active learning framework over scikit-learn; uncertainty/query-by-committee strategies. | observedstudy | Source ↗ |
| — | ntucllab/libactrepo | Pool-based active learning incl. active-learning-by-learning bandit meta-strategy. | observedstudy | Source ↗ |
| — | NUAA-AL/ALiPyrepo | Active learning toolbox with ~20 query strategies and experiment harness. | observedstudy | Source ↗ |
| — | webis-de/small-textrepo | Active learning for text classification. | observeddrop | Source ↗ |
| — | scikit-activeml/scikit-activemlrepo | sklearn-compatible active learning library, well-documented query strategies. | observedstudy | Source ↗ |
| — | SheffieldML/GPyrepo | Mature GP framework; preference/ordinal likelihoods available. | observedstudy | Source ↗ |
| — | GPflow/GPflowrepo | GPs in TensorFlow with custom likelihoods — a Bernoulli-on-latent-difference preference likelihood is straightforward. | observedstudy | Source ↗ |
| — | dragonfly/dragonflyrepo | Scalable Bayesian optimisation library. | observeddrop | Source ↗ |
| — | EmuKit/emukitrepo | Decision-making under uncertainty toolkit: BO, experimental design, sensitivity analysis. | observedstudy | Source ↗ |
| — | huawei-noah/HEBOrepo | Monorepo of BO and RL research; HEBO won the NeurIPS 2020 BBO challenge. | observedstudy — licence is fine but check the specific subdirectory | Source ↗ |
| — | optuna/optunarepo | Hyperparameter optimization framework (TPE, CMA-ES). Included for contrast: mature ask/tell ergonomics. | observedcontrast only — proves why a preference-native tool is needed | Source ↗ |
| — | Alanthink/banditpylibrepo | Lightweight bandit library including ordinary and best-arm-identification protocols. | observedstudy | Source ↗ |
| — | HuasenWu/DuelingBanditsrepo | Simulations for dueling bandit algorithms including Double Thompson Sampling. | observedreference only — do not vendor | Source ↗ |
| — | AkihikoWatanabe/DBGDrepo | Python implementation of Dueling Bandit Gradient Descent (Yue & Joachims). | observedstudy | Source ↗ |
| — | methi1999/dueling-banditsrepo | Implementations of several popular dueling bandit algorithms. | observeddrop | Source ↗ |
| — | cran/support.BWSrepo | R tools for Case 1 Best-Worst Scaling: constructs BWS questions from a BIBD and analyses responses. | observedADOPT-METHOD — best-worst is the highest-information-per-question format for 7 knobs | Source ↗ |
| — | cran/support.BWS2repo | R tools for Case 2 (profile-case) Best-Worst Scaling — best/worst ATTRIBUTE-LEVEL within a profile. | observedADOPT-METHOD — Case 2 maps almost exactly onto 'which knob is most/least right here' | Source ↗ |
| — | cran/ChoiceModelRrepo | Hierarchical Bayes multinomial logit for choice data — the shrinkage machinery that makes few-question conjoint work. | observedstudy — HB shrinkage is the key idea to port | Source ↗ |
| — | radiant-rstats/radiant.multivariaterepo | R/Shiny multivariate menu incl. conjoint analysis with a GUI. | observedreference only | Source ↗ |
| — | cran/conjointrepo | Traditional ratings-based conjoint analysis in R. | observeddrop | Source ↗ |
| — | cran/idefixrepo | Efficient (D-optimal, Bayesian) designs for discrete choice experiments, INCLUDING adaptive individual-level designs. | observedADOPT-METHOD — this is the optimal-design lane for choosing which comparisons to show | Source ↗ |
| — | cran/AlgDesignrepo | Algorithmic experimental design — D/A/I-optimal exact designs via Federov exchange. | observedstudy | Source ↗ |
| — | cran/skprrepo | Generates and evaluates optimal designs with MONTE-CARLO POWER analysis — answers 'how many trials do I need' empirically. | observedADOPT-METHOD — the simulate-to-size approach is how we should actually answer the N question | Source ↗ |
| — | cran/DoE.baserepo | Full factorials, orthogonal arrays, base DOE utilities. | observedstudy — use as the passive baseline to benchmark against | Source ↗ |
| — | pydoe/pydoerepo | Design of Experiments for Python — factorials, LHS, response surface designs. | observedstudy | Source ↗ |
| — | clicumu/pyDOE2repo | Maintained fork of pyDOE with bug fixes. | observedadopt if a Python DOE primitive is needed | Source ↗ |
| — | huggingface/trlrepo | RewardTrainer fits a Bradley-Terry reward model on pairwise preferences; DPOTrainer optimizes directly from pairs. Production-grade BT-on-preferences at scale. | observedstudy — steal the BT loss, not the trainer | Source ↗ |
| — | OpenRLHF/OpenRLHFrepo | Scalable RLHF framework (PPO/DAPO/REINFORCE) on Ray, with reward modelling from preference pairs. | observeddrop for this lane | Source ↗ |
| — | CarperAI/trlxrepo | Distributed RLHF training library. | observeddrop | Source ↗ |
| — | argilla-io/argillarepo | Collaboration tool for building high-quality labelled/preference datasets — ranking and rating question types, annotator agreement tracking. | observedstudy — good model for the annotation-UI layer and agreement metrics | Source ↗ |
| — | argilla-io/distilabelrepo | Synthetic data + AI feedback pipelines, incl. UltraFeedback-style preference generation. | observedstudy — relevant only if we bootstrap with an AI judge | Source ↗ |
| — | lm-sys/FastChatrepo | Release repo for Vicuna and Chatbot Arena; contains the Arena Elo / Bradley-Terry rating computation used on real human pairwise votes at scale. | observedstudy — the arena BT implementation is a battle-tested reference | Source ↗ |
| — | lmarena/arena-hard-autorepo | Automatic benchmark with BT modelling and bootstrap confidence intervals over pairwise judgments. | observedstudy — the bootstrap CI approach is directly reusable for reporting taste-model uncertainty | Source ↗ |
| — | christophschuhmann/improved-aesthetic-predictorrepo | CLIP+MLP aesthetic score predictor (LAION) trained on human aesthetic ratings. | observedadopt as a FEATURE — warm-start prior, then learn the client's deviation from it | Source ↗ |
| — | zai-org/ImageRewardrepo | NeurIPS 2023 general-purpose text-to-image human preference reward model, trained on 137k expert comparisons. | observedstudy — evidence on how many comparisons a general preference model needed | Source ↗ |
| — | tgxs002/HPSv2repo | Human Preference Score v2 — benchmark and model for human preferences on generated images. | observedstudy | Source ↗ |
| — | yuvalkirstain/PickScorerepo | CLIP-based preference scorer trained on Pick-a-Pic (~500k pairwise human preferences over generated images). | observedstudy — the scale contrast (500k pairs for a population model) is itself the argument for personalising with a prior | Source ↗ |
| — | anthropics/hh-rlhfrepo | Human preference dataset for helpfulness/harmlessness — canonical pairwise preference data format. | observedreference — data schema only | Source ↗ |
| — | A law of comparative judgmentpaper | Founding model: comparative judgments arise from Gaussian discriminal processes; the difference of two latent normals gives P(A>B) — the Thurstone/probit choice model. | observedcite as foundation | Source ↗ |
| — | Rank Analysis of Incomplete Block Designs: I. The Method of Paired Comparisonspaper | The Bradley-Terry model: P(i beats j) = pi_i/(pi_i+pi_j) — logistic-link paired comparison, the workhorse for our knob scoring. | observedCITE — the core model | Source ↗ |
| — | Die Berechnung der Turnier-Ergebnisse als ein Maximumproblem der Wahrscheinlichkeitsrechnungpaper | Derived the BT model and its MLE 23 years before Bradley & Terry, with an iterative algorithm and convergence conditions for tournament data. | observedcite for priority | Source ↗ |
| — | Individual Choice Behavior: A Theoretical Analysispaper | The choice axiom (independence from irrelevant alternatives) — the axiomatic basis for logit/softmax choice probabilities across N-way sets. | observedCITE — and flag IIA as a real risk for N-way galleries of similar designs | Source ↗ |
| — | The Analysis of Permutationspaper | The Plackett-Luce model for full/partial rankings — sequential Luce choices without replacement. Lets one N-way ranking substitute for many pairs. | observedCITE — the model for N-way ranking questions | Source ↗ |
| — | MM algorithms for generalized Bradley-Terry modelspaper | Minorization-maximization algorithms for BT and generalizations (ties, home-field, Plackett-Luce), with convergence guarantees. The standard practical fitting method. | observedCITE — the fitting algorithm, and the connectivity condition matters for our design | Source ↗ |
| — | Rank Centrality: Ranking from Pair-wise Comparisonspaper | Spectral rank aggregation via the stationary distribution of a random walk on the comparison graph; finite-sample error bounds for BTL scores. Sample requirement depends on the SPECTRAL GAP of the comparison graph; near order-optimal when each item is compared to randomly chosen others. | observedCITE — the 'which comparisons' structure result | Source ↗ |
| — | Spectral MLE: Top-K Rank Aggregation from Pairwise Comparisonspaper | Characterizes minimax limits for top-K identification under BTL: minimum sample complexity scales INVERSELY with the separation between the Kth and (K+1)th scores, irrespective of other distribution metrics. | observedCITE — the separation-measure insight explains why near-tied knob settings cost disproportionately many comparisons | Source ↗ |
| — | Top-K Ranking from Pairwise Comparisons: When Spectral Ranking is Optimalpaper | Upper and lower sample-size bounds for reliable top-K recovery; under a random comparison model the gap narrows to a constant factor, so a spectral method alone attains minimax-optimal sample complexity without a second MLE stage. | observedcite | Source ↗ |
| — | Active Ranking from Pairwise Comparisons and when Parametric Assumptions Don't Helppaper | Adaptive algorithm counting comparisons won, with confidence-interval stopping, achieves ranking with a number of comparisons OPTIMAL UP TO LOGARITHMIC FACTORS and requires NO structural assumption on the comparison-probability matrix. Crucially proves a lower bound for parametric models showing BTL/Thurstone assumptions buy AT MOST LOGARITHMIC GAINS for stochastic comparisons. | observedCITE — HIGH IMPORTANCE: this is the honest counterweight to assuming BTL saves us a lot of questions | Source ↗ |
| — | Simple, Robust and Optimal Ranking from Pairwise Comparisonspaper | The Copeland counting algorithm (rank by number of comparisons won) is optimal up to CONSTANT factors for top-k recovery, imposes no conditions on the comparison-probability matrix, and is orders of magnitude faster than prior methods. | observedCITE — argues a trivially simple estimator may suffice; a strong simplicity check on our design | Source ↗ |
| — | Active Ranking using Pairwise Comparisonspaper | THE CRITICAL RESULT. Objects embedded in d-dimensional Euclidean space, ranked by distance from a common reference point: the number of possible rankings grows like n^(2d), and an algorithm identifies a randomly selected ranking using JUST SLIGHTLY MORE THAN d log n ADAPTIVELY selected pairwise comparisons, ON AVERAGE. If comparisons are chosen at random instead, ALMOST ALL pairwise comparisons must be made. Also gives an error-tolerant variant requiring only that comparisons are probably correct. | observedCITE FIRST — directly licenses the 'few comparisons over 7 knobs' claim, WITH its average-case caveat | Source ↗ |
| — | An Active Learning Algorithm for Ranking from Pairwise Preferences with an Almost Optimal Query Complexitypaper | Active learning for ranking under possible NON-TRANSITIVITY from human error/irrationality; query bounds significantly beat non-active bounds for the same error guarantee, almost achieving the information-theoretic lower bound. | observedCITE — the tolerance for intransitive human answers is essential for real taste data | Source ↗ |
| — | Noisy Sorting Without Resamplingpaper | With each comparison correct with probability at least 1/2+gamma and NO re-querying of the same pair, an algorithm with running time n^O(gamma^-4) and SAMPLING COMPLEXITY O_gamma(n log n) recovers the order w.h.p. The recovered order sigma satisfies sum_i|sigma(i)-pi(i)| = Theta(n) and max_i|sigma(i)-pi(i)| = Theta(log n). | observedCITE — sets the noisy passive baseline of n log n that adaptivity must beat | Source ↗ |
| — | Minimax-optimal Inference from Partial Rankingspaper | Minimax error rates for estimating BTL/PL parameters from partial rankings; quantifies the information gained per ranking of a set of size k. | observedcite — supports N-way ranking over pairs on information grounds | Source ↗ |
| — | When is it Better to Compare than to Score?paper | Directly analyses the cardinal (rating) vs ordinal (comparison) tradeoff — when pairwise comparison beats direct scoring. | observedCITE — directly on our 'should we ask for ratings or comparisons' decision; READ FULL TEXT BEFORE QUOTING | Source ↗ |
| — | On a Measure of the Information Provided by an Experimentpaper | Expected information gain (expected KL from prior to posterior) as the criterion for experiment value — the formal basis for choosing the most informative next comparison. | observedCITE — the objective function for query selection | Source ↗ |
| — | Information-Based Objective Functions for Active Data Selectionpaper | Practical information-based objectives for active data selection in Bayesian models; the bridge from Lindley's theory to usable acquisition functions. | observedCITE | Source ↗ |
| — | Bayesian Experimental Design: A Reviewpaper | Canonical review unifying Bayesian design criteria (D-, A-, E-optimality) under a decision-theoretic utility framework. | observedCITE — the reference for D-optimality framing | Source ↗ |
| — | Bayesian Active Learning for Classification and Preference Learningpaper | BALD: expresses information gain in terms of PREDICTIVE ENTROPIES, making it tractable for GP classifiers; explicitly EXTENDS TO GAUSSIAN PROCESS PREFERENCE LEARNING by reformulating binary preference learning as classification. Equal or lower computational cost than competitors. | observedADOPT — this is the acquisition function to implement for pairwise queries | Source ↗ |
| — | Preference learning with Gaussian processespaper | GP prior over a latent preference function with a Thurstone-style likelihood on pairwise comparisons; Laplace approximation for inference. The foundation of BoTorch's PairwiseGP. | observedCITE — the model behind PairwiseGP | Source ↗ |
| — | Active Preference Learning with Discrete Choice Datapaper | GP preference model + expected-improvement acquisition over gallery queries. USER STUDY (5 subjects, 50 trials each arm, 38 MERL BRDFs): maxEI needed a MEAN OF 8.56 +/- 5.23 clicks to find a target, versus 17.87 +/- 8.60 for max-variance and 18.40 +/- 7.87 for Latin hypercubes. Seeded with 4 predetermined queries. Authors state 'requiring more than 50 user queries in a real application would be unacceptable'. | observedCITE — best hard number for 'how many clicks', with the caveat that 6D already strained it | Source ↗ |
| — | Preferential Bayesian Optimizationpaper | Formalises PBO: optimize a latent function queried only through pairwise duels, modelling duel outcomes with a GP + Bernoulli likelihood; introduces acquisition functions incl. the dueling-Thompson approach. Reports PBO 'needs drastically fewer comparisons for finding the optimum', with correlation modelling identified as the key advantage. | observedCITE — the framework paper; do not quote a number from it | Source ↗ |
| — | qEUBO: A Decision-Theoretic Acquisition Function for Preferential Bayesian Optimizationpaper | qEUBO (expected utility of the best option) is ONE-STEP BAYES OPTIMAL under noise-free responses (equivalent to knowledge gradient), has an additive-constant approximation guarantee under noisy responses, and its Bayesian simple regret converges to zero at rate o(1/n). Shows qEI can FAIL to converge for PBO. | observedADOPT — the acquisition function to use, already in BoTorch (R14/R16) | Source ↗ |
| — | Sequential Line Search for Efficient Visual Design Optimization by Crowdspaper | CLOSEST PRIOR ART #1. Preferential BO where each query is a 1D slider through high-dim design space. ALL REPORTED RESULTS USED 15 ITERATIONS, 7 crowd microtasks per iteration, proceeding once >=5 responses arrived (total crowd cost 5.25 USD per result). Convergence observation: 'the distances become small rapidly in the first 4 or 5 iterations'. Design spaces: 6D photo colour enhancement, 3D and 7D BRDF material design; synthetic tests at n in {2,6,20}. | observedCITE FIRST among applied work — 7D is our exact dimensionality, and 'good by 4-5 iterations' is the most directly transferable number | Source ↗ |
| — | Sequential Gallery for Interactive Visual Design Optimizationpaper | CLOSEST PRIOR ART #2. Sequential plane search: user picks best from a 2D grid gallery each round. USER STUDY: 'The mean iteration count necessary for finding satisfactory results was 5.36 with SD = 2.69'; 'One plane-search subtask took 14.8 seconds on average'; participants were asked to continue for 15 iterations regardless. 5 of 6 participants hit satisfaction within 15. Synthetic functions at 5D, 15D, 10D, 20D. Grid: 25 points in a 5-by-5 lattice for photo enhancement, reduced to 3-by-3 for body-shape design on a 13-inch display. | observedCITE FIRST — 5.36 +/- 2.69 rounds at ~15s each is the single most quotable applied figure for our deliverable | Source ↗ |
| — | A Bayesian Interactive Optimization Approach to Procedural Animation Designpaper | Gallery-based interactive BO for animation design. Table 1: 4-gallery on 4 parameters needed 7.57 +/- 4.67 iterations; PFHP pairwise on 4 params 8.45 +/- 2.81; the full 12-parameter task with 4-gallery+manual needed 5.38 +/- 2.63 iterations versus 28.40-35.33 for manual tuning. A learned prior cut iterations 'from an average of 11.25 to just 6.5'. Authors adopted a 20-iteration budget 'because this is roughly the point at which users start to quit if they do not see significant improvement'. | observedCITE — the 20-iteration patience ceiling is a hard UX constraint for our design | Source ↗ |
| — | Interactively optimizing information retrieval systems as a dueling bandits problempaper | Introduces Dueling Bandit Gradient Descent — online optimization from relative (pairwise) feedback only. | observedcite | Source ↗ |
| — | The K-armed dueling bandits problempaper | Formalises the K-armed dueling bandit problem and gives the Interleaved Filter algorithm with regret bounds under relative feedback. | observedcite | Source ↗ |
| — | Relative Upper Confidence Bound for the K-Armed Dueling Bandit Problempaper | RUCB: the first dueling-bandit algorithm not requiring knowledge of the time horizon or an explicit exploration phase. | observedcite | Source ↗ |
| — | Advancements in Dueling Banditspaper | Survey of dueling bandit formulations, algorithms and applications — the orientation map for relative-feedback methods. | observedcite — read first for orientation | Source ↗ |
| — | TrueSkill: A Bayesian Skill Rating Systempaper | Factor-graph Bayesian skill rating with per-item Gaussian uncertainty updated by expectation propagation — gives calibrated uncertainty per design, which Elo does not. | observedCITE — the uncertainty-tracking idea; implement via openskill.py not sublee/trueskill | Source ↗ |
| — | TrueSkill 2: An improved Bayesian skill rating systempaper | Reported extension of TrueSkill handling team correlations and individual statistics. | hypothesisHOLD — verify before use | Source ↗ |
| — | Some probabilistic models of best, worst, and best-worst choicespaper | Axiomatic probabilistic models for best-worst choice, relating BW choice probabilities to underlying scale values. | observedCITE — the theoretical basis for MaxDiff | Source ↗ |
| — | Probabilistic models of set-dependent and attribute-level best-worst choicepaper | Extends BW models to ATTRIBUTE-LEVEL (Case 2) best-worst — choosing the best and worst attribute-level within a single profile. | observedCITE — Case 2 is the format that maps onto 'which knob is most/least right on this design' | Source ↗ |
| — | Best-worst scaling: What it can do for health care research and how to do itpaper | Practical guide to designing and analysing BWS studies. | observedCITE for method; DO NOT quote an information-gain multiplier without reading full text | Source ↗ |
| — | Best-Worst Scaling: Theory, Methods and Applicationspaper | The canonical book-length treatment of BWS Cases 1-3. | observedcite — the reference for BWS design | Source ↗ |
| — | Conjoint Analysis in Consumer Research: Issues and Outlookpaper | The foundational review establishing conjoint analysis — decomposing overall preference judgments into attribute part-worths. Our 7 knobs are a conjoint attribute set. | observedCITE — the framing ancestor of our problem | Source ↗ |
| — | Fast Polyhedral Adaptive Conjoint Estimationpaper | THE KEY ADAPTIVE-CONJOINT RESULT. Explicitly targets q < p (fewer questions than parameters): 'We anticipate that the polyhedral methods are particularly well-suited to solving problems in which there are a large number of parameters relative to the number of responses from each individual (q < p). Thus, we vary the number of questions from slightly less than the number of parameters (q = 8) to comfortably more than the number of parameters (q = 16).' With p = 10 parameters. Abstract: 'For low numbers of questions, polyhedral question design does best (or is tied for best) for all tested domains. For high numbers of questions, efficient fixed designs do better in some domains.' Field test: 330 respondents, 16 paired-comparison questions + 4 holdouts, 9 binary features + price. | observedCITE — the strongest published evidence that adaptive design works with FEWER questions than parameters | Source ↗ |
| — | Polyhedral Methods for Adaptive Choice-Based Conjoint Analysispaper | Choice-based (not metric) polyhedral adaptive conjoint. Simulation used q = 16 questions on 4 features x 4 levels x 4 profiles (12 free part-worths), with q = 8 and q = 24 giving 'similar qualitative insights'; 1000 simulated respondents. Field application: 354 web respondents, 8 multi-level program features, and 'Pretests confirmed that respondents could comfortably answer 12 stated-choice questions'. | observedCITE — '12 choice questions for 8 multi-level features' is the most transferable applied number | Source ↗ |
| — | The magical number seven, plus or minus two: Some limits on our capacity for processing informationpaper | The famous 7+/-2. WHAT IT ACTUALLY SUPPORTS: limits on ABSOLUTE JUDGMENT of unidimensional stimuli and on IMMEDIATE MEMORY span for chunks. Miller himself treats the recurrence of ~7 across these distinct paradigms with explicit irony. | observedCITE ONLY WITH THE CORRECTION — flagging the misuse is more valuable to the client than repeating it | Source ↗ |
| — | When choice is demotivating: Can one desire too much of a good thing?paper | The jam study. Study 1: a tasting booth displayed either a limited (6) or extensive (24) set of jams. 'Nearly 30% (31) of the consumers in the limited-choice condition subsequently purchased a jar... in contrast, only 3% (4) of the consumers in the extensive-choice condition did so, X2(1, N = 249) = 32.34, p < .0001.' But ATTRACTION ran the opposite way: 60% of passers-by stopped at the extensive display vs 40% at the limited one. Study 2 used 6 vs 30 essay topics (the '30' in the abstract is NOT the jam study). | observedCITE ONLY PAIRED WITH P43 — citing this alone as a design principle will not survive scrutiny | Source ↗ |
| — | Can There Ever Be Too Many Options? A Meta-Analytic Review of Choice Overloadpaper | DECISIVE REPLICATION EVIDENCE. 'In a meta-analysis of 63 conditions from 50 published and unpublished experiments (N = 5,036), we found a mean effect size of virtually zero but considerable variance between studies.' Exact: 'The mean effect size of choice overload across all 63 data points... is D = 0.02 (95% confidence interval [CI95] -0.09 to 0.12).' Trimmed 20%: 'Dtrimmed = 0.001 (CI95 -0.08 to 0.07).' Heterogeneity I2 = 68% untrimmed, 22% trimmed. Conclusion: 'no sufficient conditions could be identified' and 'adverse consequences due to having too much choice are not a robust phenomenon'. Also notes a slight publication bias. | observedCITE — ANSWERS THE REPLICATION QUESTION: choice overload does NOT robustly replicate; gallery size should be chosen on information grounds, not overload fear | Source ↗ |
| — | The construction of preferencepaper | Preferences are frequently CONSTRUCTED during elicitation rather than retrieved from a stable store — they are sensitive to framing, response mode and context. | observedCITE — THE key epistemic caveat: the elicitation procedure partly CREATES the taste it measures, which bounds how much precision is even meaningful | Source ↗ |
| — | Attitudinal effects of mere exposurepaper | Repeated exposure to a stimulus increases liking for it — mere exposure. | observedCITE — a genuine confound: designs shown repeatedly across elicitation rounds may gain preference from exposure alone, biasing later comparisons | Source ↗ |
| — | Processing Fluency and Aesthetic Pleasure: Is Beauty in the Perceiver's Processing Experience?paper | Aesthetic pleasure is substantially a function of PROCESSING FLUENCY — stimuli that are easier to process are experienced as more beautiful; fluency is affected by symmetry, contrast, prototypicality and prior exposure. | observedCITE — explains WHY the knobs are correlated and why a low-dimensional latent structure (the Jamieson-Nowak precondition) is plausible for visual taste | Source ↗ |
| — | The role of visual complexity and prototypicality regarding first impression of websitespaper | Visual complexity and prototypicality jointly drive first impressions of websites, with effects detectable at very short exposures. | observedcite for the two-factor structure only | Source ↗ |
| — | Predicting users' first impressions of website aesthetics with a quantification of perceived visual complexity and colorfulnesspaper | EXPOSURE CONFIRMED AT 500ms. Computational models of colourfulness (r = .88, R2 = .78, adj R2 = .77) and visual complexity (r = .80, R2 = .65, adj R2 = .64) against mean human ratings of those constructs. The APPEAL model: 'The model accounts for 48% of the variation in aesthetic preferences (adj. R2 = .48)' after 500ms viewing. Data: 548 volunteers rating 450 websites (184 colourfulness / 122 complexity / 242 appeal); 421 pages analysed for colourfulness, 382 for complexity. | observedCITE — two computed features + demographics explain ~48% of first-impression variance; the residual ~52% is roughly the space a per-client taste model must earn its keep in | Source ↗ |
| — | Attention web designers: You have 50 milliseconds to make a good first impression!paper | Three studies. Studies 1-2 presented homepages for 500ms; Study 3 added a 50ms condition on the same stimuli. Abstract states 'visual appeal ratings were highly correlated from one phase to the next as were the correlations between the 50 ms and 500 ms conditions', concluding visual appeal can be assessed within 50ms. | observedCITE THE QUALITATIVE CLAIM ONLY — explicitly do not quote a coefficient | Source ↗ |
| — | Deep reinforcement learning from human preferencespaper | Learns a reward model from human pairwise comparisons of trajectory segments, demonstrating complex behaviours from feedback on well under 1% of agent interactions. | observedcite — the modern proof that pairwise human feedback is a viable training signal | Source ↗ |
| 1 | Sequential Gallery (Koyama, Sato, Goto 2020)paper | CLOSEST PRIOR ART #2. Sequential plane search: user picks best from a 2D grid gallery each round. USER STUDY: 'The mean iteration count necessary for finding satisfactory results was 5.36 with SD = 2.69'; 'One plane-search subtask took 14.8 seconds on average'; participants were asked to continue for 15 iterations regardless. 5 of 6 participants hit satisfaction within 15. Synthetic functions at 5D, 15D, 10D, 20D. Grid: 25 points in a 5-by-5 lattice for photo enhancement, reduced to 3-by-3 for body-shape design on a 13-inch display. | observedCITE FIRST — 5.36 +/- 2.69 rounds at ~15s each is the single most quotable applied figure for our deliverable | Source ↗ |
| 2 | Jamieson & Nowak, Active Ranking using Pairwise Comparisons (NIPS 2011)paper | THE CRITICAL RESULT. Objects embedded in d-dimensional Euclidean space, ranked by distance from a common reference point: the number of possible rankings grows like n^(2d), and an algorithm identifies a randomly selected ranking using JUST SLIGHTLY MORE THAN d log n ADAPTIVELY selected pairwise comparisons, ON AVERAGE. If comparisons are chosen at random instead, ALMOST ALL pairwise comparisons must be made. Also gives an error-tolerant variant requiring only that comparisons are probably correct. | observedCITE FIRST — directly licenses the 'few comparisons over 7 knobs' claim, WITH its average-case caveat | Source ↗ |
| 3 | meta-pytorch/botorch (MIT, 3589 stars)repo | Bayesian optimization in PyTorch. Ships PairwiseGP, PairwiseLaplaceMarginalLogLikelihood, and qEUBO — a production-grade preferential-BO stack. | observedADOPT — the single most load-bearing repo for this lane | Source ↗ |
| 4 | Sequential Line Search (Koyama et al. SIGGRAPH 2017)paper | CLOSEST PRIOR ART #1. Preferential BO where each query is a 1D slider through high-dim design space. ALL REPORTED RESULTS USED 15 ITERATIONS, 7 crowd microtasks per iteration, proceeding once >=5 responses arrived (total crowd cost 5.25 USD per result). Convergence observation: 'the distances become small rapidly in the first 4 or 5 iterations'. Design spaces: 6D photo colour enhancement, 3D and 7D BRDF material design; synthetic tests at n in {2,6,20}. | observedCITE FIRST among applied work — 7D is our exact dimensionality, and 'good by 4-5 iterations' is the most directly transferable number | Source ↗ |
| 5 | Scheibehenne, Greifeneder & Todd 2010 choice-overload meta-analysispaper | DECISIVE REPLICATION EVIDENCE. 'In a meta-analysis of 63 conditions from 50 published and unpublished experiments (N = 5,036), we found a mean effect size of virtually zero but considerable variance between studies.' Exact: 'The mean effect size of choice overload across all 63 data points... is D = 0.02 (95% confidence interval [CI95] -0.09 to 0.12).' Trimmed 20%: 'Dtrimmed = 0.001 (CI95 -0.08 to 0.07).' Heterogeneity I2 = 68% untrimmed, 22% trimmed. Conclusion: 'no sufficient conditions could be identified' and 'adverse consequences due to having too much choice are not a robust phenomenon'. Also notes a slight publication bias. | observedCITE — ANSWERS THE REPLICATION QUESTION: choice overload does NOT robustly replicate; gallery size should be chosen on information grounds, not overload fear | Source ↗ |
| 6 | Heckel, Shah, Ramchandran & Wainwright, Active Ranking / parametric assumptionspaper | Adaptive algorithm counting comparisons won, with confidence-interval stopping, achieves ranking with a number of comparisons OPTIMAL UP TO LOGARITHMIC FACTORS and requires NO structural assumption on the comparison-probability matrix. Crucially proves a lower bound for parametric models showing BTL/Thurstone assumptions buy AT MOST LOGARITHMIC GAINS for stochastic comparisons. | observedCITE — HIGH IMPORTANCE: this is the honest counterweight to assuming BTL saves us a lot of questions | Source ↗ |
| 7 | Toubia et al., Fast Polyhedral Adaptive Conjoint Estimation (2003)paper | THE KEY ADAPTIVE-CONJOINT RESULT. Explicitly targets q < p (fewer questions than parameters): 'We anticipate that the polyhedral methods are particularly well-suited to solving problems in which there are a large number of parameters relative to the number of responses from each individual (q < p). Thus, we vary the number of questions from slightly less than the number of parameters (q = 8) to comfortably more than the number of parameters (q = 16).' With p = 10 parameters. Abstract: 'For low numbers of questions, polyhedral question design does best (or is tied for best) for all tested domains. For high numbers of questions, efficient fixed designs do better in some domains.' Field test: 330 respondents, 16 paired-comparison questions + 4 holdouts, 9 binary features + price. | observedCITE — the strongest published evidence that adaptive design works with FEWER questions than parameters | Source ↗ |
| 8 | Houlsby et al., BALD (2011)paper | BALD: expresses information gain in terms of PREDICTIVE ENTROPIES, making it tractable for GP classifiers; explicitly EXTENDS TO GAUSSIAN PROCESS PREFERENCE LEARNING by reformulating binary preference learning as classification. Equal or lower computational cost than competitors. | observedADOPT — this is the acquisition function to implement for pairwise queries | Source ↗ |
| 9 | Slovic, The Construction of Preference (1995)paper | Preferences are frequently CONSTRUCTED during elicitation rather than retrieved from a stable store — they are sensitive to framing, response mode and context. | observedCITE — THE key epistemic caveat: the elicitation procedure partly CREATES the taste it measures, which bounds how much precision is even meaningful | Source ↗ |
| 10 | Brochu, de Freitas & Ghosh, Active Preference Learning with Discrete Choice Data (NIPS 2007)paper | GP preference model + expected-improvement acquisition over gallery queries. USER STUDY (5 subjects, 50 trials each arm, 38 MERL BRDFs): maxEI needed a MEAN OF 8.56 +/- 5.23 clicks to find a target, versus 17.87 +/- 8.60 for max-variance and 18.40 +/- 7.87 for Latin hypercubes. Seeded with 4 predetermined queries. Authors state 'requiring more than 50 user queries in a real application would be unacceptable'. | observedCITE — best hard number for 'how many clicks', with the caveat that 6D already strained it | Source ↗ |
110 mined records. Search and inspect the full agent output; nothing is collapsed into a top-ten headline.
| Rank | Identity | What the agent found | Evidence | Source |
|---|---|---|---|---|
| — | Whole-screen stimuli with only targeted knobs variedinnovation | Render complete realistic screens but vary only the knobs the round targets, holding others at the posterior mean. | inferredADOPT | D-optimal / Bayesian adaptive DCE design (idefix); live demo already does this |
| — | Content held constant across a comparisoninnovation | Use the same headline/copy/imagery in all cards of a round so content preference cannot confound the token-pack signal. | inferredADOPT | basic experimental control |
| — | P05 corpus components as re-themable controlled stimuliinnovation | Use the 8,515-identity corpus as the stimulus pool: identical structure, varying token pack, re-theming is a find-and-replace on ~30 custom properties. | observedADOPT | research/21st-corpus-audit-2026-08-27.md — 86.7% of colour-bearing CSS rules already resolve through var(--token); 87.3% carry a shadcn oklch :root block |
| — | Stimulus realism gradient: abstract swatches -> component -> full screeninnovation | Early rounds use cheap abstract stimuli, later rounds full screens. | hypothesistest | none |
| — | Never re-show an identical stimulusinnovation | Prevents familiarity inflating preference for repeated designs. | inferredADOPT | Zajonc 1968 mere exposure |
| — | Log stimulus-repetition rate as a measured confoundinnovation | Even if repeats occur, make the exposure effect measurable rather than invisible. | inferredADOPT | Zajonc 1968 |
| — | Industry-conditioned stimulus priorsinnovation | Seed the stimulus space from an industry prior (a law firm starts in a more conservative region than a course creator). | inferredADOPT | b2b-template-shelf-report.md 17-industry variant deltas |
| — | Brand-constraint hard cutoffs before elicitationinnovation | Existing brand colours/fonts become hard prunes of the stimulus space, not soft preferences; all later stimuli satisfy them. | observedADOPT | Sawtooth ACBC must-have/unacceptable mechanism (vendor manual) |
| — | Dark-mode variant shown as part of the stimulusinnovation | Show each candidate in both modes since packs must ship both. | inferredtest | token-pack-science §3.9 — dark is not inversion |
| — | Density stress-test stimuli using realistic long copyinnovation | Include a card rendered with realistically long client copy so density preference is measured under real conditions. | inferredtest | ui-pick-to-spec §6 failure mode 6 |
| — | Fragment rounds reserved for fine disambiguation onlyinnovation | Use component fragments only when two knobs remain confounded at whole-screen scale. | hypothesistest — this is falsifier F4 | none |
| — | Adaptive grid size by viewportinnovation | Gallery size should adapt to display, not be a fixed product constant. | observedADOPT | Sequential Gallery reduced 5x5 to 3x3 on a 13-inch display |
| — | Stimuli drawn from the closed pack catalogue onlyinnovation | Never show a card that is not a real, gate-passing pack — so the preview IS the deliverable. | inferredADOPT | token-pack-science gates A-J |
| — | Preview must be re-rendered from spec, never the original thumbnailinnovation | Closes the expectation gap by construction and makes previews a free end-to-end pipeline test. | observedADOPT | ui-pick-to-spec §6 operational consequence |
| — | Anti-prototypicality probeinnovation | Deliberately include one low-prototypicality card per round to detect clients who want distinctive over safe. | inferredtest | Tuch et al. 2012; Reber et al. 2004 |
| — | Seed from client's existing website via extractioninnovation | Warm-start the prior from the client's current site rather than a uniform prior. | observedtest | Canva AI extracts brand context from a public URL |
| — | Competitor-anchored stimuliinnovation | Show packs near and far from named competitors to elicit differentiation preference. | hypothesistest | none |
| — | Stimulus set diversity floorinnovation | Enforce a minimum pairwise distance among the 4 cards so the comparison graph stays well-connected. | inferredADOPT with tension noted | Negahban et al. spectral gap dependence |
| — | 4-up gallery with explicit outside optioninnovation | 2.32 noiseless bits/round; ~0.91 effective at p=0.15; balances information against deliberation cost and avoids the hostility of 'worst'. | inferredADOPT — primary recommendation | derived §3 + Sawtooth 4-5 per screen + live demo |
| — | Best-worst of 4 (MaxDiff Case 1)innovation | 3.58 noiseless bits/round, ~2x the 4-up rate; would cut rounds to ~3. | observedtest as an A/B arm — falsifier F6 | Marley & Louviere 2005; Sawtooth manual |
| — | Attribute-level best-worst (MaxDiff Case 2)innovation | 'Which knob is most and least right on THIS design' — maps almost exactly onto per-knob credit assignment, solving the attribution problem a whole-card pick has. | inferredtest — high upside, high risk | Marley, Flynn & Louviere 2008; support.BWS2 |
| — | Binary pairwiseinnovation | Simplest, but ~11 rounds at p=0.15 and most sensitive to near-ties. | inferredreject as primary | derived §3 |
| — | Three-way pairwise with a neutral middleinnovation | A shipped outside-option affordance inside a pairwise format. | observedreference — the affordance, not the format | Midjourney legacy Style Tuner: 'leave the middle box selected to skip the pair' |
| — | Full ranking of 4innovation | 4.58 noiseless bits, the densest format tested. | inferredreject | Plackett 1975; Hajek, Oh & Xu 2014 |
| — | 9-up galleryinnovation | 3.17 noiseless bits. | inferredreject for whole screens | derived §3; Sequential Gallery used 5x5 and 3x3 |
| — | Mixed format by phase: 4-up early, pairwise lateinnovation | Broad exploration wants breadth; fine disambiguation is naturally pairwise. | hypothesistest | none |
| — | Slider direct manipulation as a fallbackinnovation | Offer sliders to clients who DO know what they want, skipping elicitation entirely. | observedADOPT as escape hatch | Figma First Draft radius/spacing sliders |
| — | Sequential line search (1D slider through knob space)innovation | User picks a point on a 1D slice; reported 15-iteration budget, good by iteration 4-5 at 6D/7D. | observedstudy | Koyama et al. SIGGRAPH 2017 (MIT-licensed implementation exists) |
| — | Outside option as weak negative evidence on all shown cardsinnovation | Preserves information from declines while keeping the re-roll honest. | hypothesisADOPT — calibrate the weight | derived; diverges from Midjourney which makes skipping non-informative |
| — | Cap consecutive outside-option selections at 3innovation | Three declines in a row means the generator is in the wrong region; trigger a re-seed rather than another round. | hypothesisADOPT | derived |
| — | 'This is right' always-available terminal affordanceinnovation | User's own judgement is the criterion the whole exercise proxies for. | observedADOPT | Sequential Gallery satisfaction button; mean 5.36 rounds to satisfaction |
| — | Preview-before-commit on the selected packinnovation | Let the client see the pack applied to their real content before committing. | observedADOPT | Webflow 'Preview in Designer' |
| — | Paired-comparison tie option distinct from 'none'innovation | 'These two are equally good' is different information from 'both are wrong'. | inferredtest | prefmod handles undecided/ties |
| — | Confidence-weighted responses (client marks how sure they are)innovation | Let the client flag a low-confidence pick so it is down-weighted. | hypothesisreject | none |
| — | BALD acquisition on a GP preference modelinnovation | Information gain expressed via predictive entropies, explicitly extended to GP preference learning; tractable. | observedADOPT | Houlsby et al. 2011 (arXiv 1112.5745) |
| — | qEUBO acquisitioninnovation | One-step Bayes optimal under noise-free responses; simple regret converges at o(1/n); qEI can FAIL to converge for PBO. | observedADOPT | Astudillo et al. AISTATS 2023; shipped in BoTorch |
| — | Target the least-resolved knob each roundinnovation | Constructs cards differing on the widest-posterior axis while holding others steady. | observedADOPT | live demo already does this |
| — | Explicit exploration/exploitation schedule ('candy vs medicine')innovation | Deliberately mix near-certain-hit cards (engagement) with high-information cards (learning). | observedADOPT — calibrate ratio | Stitch Fix Style Shuffle practice (press-reported) |
| — | Hierarchical Bayes shrinkage toward a population priorinnovation | Designed exactly for the few-observations-per-individual regime; the primary anti-overfitting mechanism alongside closed packs. | observedADOPT — but cold-start it from I40 | ChoiceModelR; standard conjoint practice |
| — | Warm-start prior from a population aesthetic modelinnovation | Learn the client's DEVIATION from a population baseline rather than their taste from scratch. The largest single efficiency lever reported anywhere in the applied literature. | observedADOPT | LAION aesthetic predictor; Brochu et al. 2010 cut iterations 11.25 -> 6.5 with a learned prior |
| — | D-optimal / Bayesian-efficient stimulus set selectioninnovation | Choose the 4 cards to maximise design efficiency under the current posterior. | observedADOPT-METHOD, reimplement | idefix (R, GPL-3) |
| — | Simulate-to-size: Monte Carlo power analysis for round countinnovation | Answer 'how many rounds' EMPIRICALLY by simulating synthetic clients under a BT likelihood rather than arguing from bounds. | observedADOPT — this is how the N question should actually be settled | skpr (R) |
| — | Orthogonal-array fixed design as the passive baselineinnovation | The non-adaptive control our adaptive policy must beat. | observedADOPT as control arm | DoE.base |
| — | Projective preferential BO for high-dim knob spacesinnovation | Queries along projections, designed for human-in-the-loop high-dimensional elicitation. | observedstudy | AaltoPML/PPBO (MIT) |
| — | Transitivity-based pair elimination (PAPRIKA-style)innovation | Eliminate implied comparisons by transitivity, drastically cutting explicit questions. | hypothesisVERIFY FIRST, then test | 1000minds method; COULD NOT VERIFY — 404 on both URLs |
| — | Non-transitivity-tolerant active rankinginnovation | Explicitly handles 'non-transitivity paradoxes which may arise naturally due to human mistakes or irrationality'. | observedstudy | Ailon 2010 (arXiv 1011.0108) |
| — | Copeland counting as a simplicity checkinnovation | Rank by comparisons won: optimal up to CONSTANT factors, no conditions on the probability matrix. If this matches the GP model's output, the GP is not earning its complexity. | observedADOPT as a baseline check | Shah & Wainwright 2015 |
| — | Dueling-bandit formulationinnovation | Relative-feedback online optimisation. | observedreject as primary framing | Yue & Joachims; Zoghi RUCB; Sui et al. survey |
| — | Per-knob independent BT models vs joint GPinnovation | Independent per-knob models are simpler and interpretable; a joint GP captures correlation. | hypothesistest — joint expected to win | derived |
| — | Bootstrap confidence intervals over pairwise judgmentsinnovation | Battle-tested approach to reporting uncertainty on BT fits from human votes. | observedADOPT | lmarena/arena-hard-auto |
| — | TrueSkill-style Gaussian belief per packinnovation | Calibrated per-item uncertainty that Elo does not give. | observedstudy — openskill.py only | Herbrich et al. 2007; use openskill.py (MIT), NOT sublee/trueskill |
| — | Stop when expected information gain of the best next question < thresholdinnovation | A wide posterior no available question can narrow is a reason to stop, not to continue. | hypothesisADOPT | derived |
| — | Three-condition stop: confidence OR ceiling OR user-satisfiedinnovation | Primary rule is confidence; 15 rounds is a ceiling; user satisfaction always terminates. | inferredADOPT | derived §6; Brochu 2010 20-iteration abandonment; Midjourney 'past round 15... small and subtle' |
| — | Hard floor of 5 roundsinnovation | Early apparent convergence is usually the prior, not the data. | inferredADOPT | derived; Sequential Gallery mean 5.36 |
| — | Per-knob confidence reporting, never a single scalarinnovation | Some knobs resolve in two rounds; others may never resolve, and 'this client does not care about shadow' is a finding not a failure. A single '87% confident' number would mislead. | hypothesisADOPT | derived |
| — | 'No preference' as a first-class per-knob outcomeinnovation | Explicitly represent indifference rather than forcing a point estimate. | hypothesisADOPT | derived |
| — | Two-week test-retest stability as the primary validation metricinnovation | A model that fits the clicks but changes answer next Tuesday is worthless. Measures whether the thing we claim to measure exists. | inferredADOPT — this is falsifier F7 | Slovic 1995 construction of preference |
| — | Held-out pick prediction as the fit metricinnovation | Reserve 2 rounds, predict them from the model fitted on the rest. | hypothesisADOPT | derived |
| — | Neighbour-discrimination testinnovation | Show the client their pack plus the 2nd and 3rd nearest, unlabelled. If they cannot pick their own above chance, the catalogue is denser than perception. | hypothesisADOPT — falsifier F5 | derived |
| — | Perturbation-tolerance testinnovation | Show the converged pack and a +/-1-level perturbation. If clients reliably distinguish and reject, the tolerance argument in §2 collapses and round counts triple. | hypothesisADOPT — falsifier F1, run FIRST | derived |
| — | Separation-aware stoppinginnovation | Detect when remaining candidates are genuinely near-tied and stop rather than burning rounds distinguishing the indistinguishable. | observedADOPT | Chen & Suh 2015 — complexity scales inversely with separation |
| — | Comparison-graph connectivity checkinnovation | BT/MM fitting requires a strongly connected comparison graph; verify before fitting. | observedADOPT as a gate | Hunter 2004 MM convergence condition |
| — | Abandonment-rate monitoring as a UX stop signalinnovation | Instrument drop-off per round; if it rises before the ceiling, lower the ceiling. | observedADOPT | Brochu et al. 2010: 20 iterations is 'roughly the point at which users start to quit' |
| — | Time-per-round budgetinnovation | Budget total elicitation time, not just round count. 10 rounds x ~15s is ~2.5 minutes. | observedADOPT | Sequential Gallery: 14.8s per plane-search subtask |
| — | Confidence decay over calendar timeinnovation | Treat the profile as decaying, prompting re-elicitation rather than assuming permanence. | inferredtest | Netflix supersession rule; Hinge 24h window |
| — | Re-elicitation triggered by client rejection of built outputinnovation | If a client rejects the delivered design, that is strong evidence to re-run rather than patch. | hypothesistest | derived |
| — | Continuous preference vector internally, closed pack at the boundaryinnovation | Catalogue can grow without re-eliciting; shipped artefact is always gate-passing. | inferredADOPT — primary recommendation | derived §8; token-pack-science gates A-J; lane synthesis 'learn a vector not a pack ID' |
| — | Reject continuous token interpolationinnovation | The midpoint of two AA-passing palettes can fail AA; an interpolated pack has passed none of gates A-J. | observedADOPT the rejection | token-pack-science §5 Gate C — WCAG luminance is non-linear in channel values |
| — | Closed-pack snapping AS the anti-overfitting regularizerinnovation | Output space of ~20-30 packs rather than 36,000 means the model cannot overfit into a bespoke corner on ten noisy clicks. | hypothesisADOPT | derived §8 |
| — | Explain the pack choice in knob termsinnovation | 'We chose this because you are here in knob space' is worth real money in the client conversation. | hypothesisADOPT | derived |
| — | Weighted knob-space distance metricinnovation | Typography and chroma likely dominate perception; equal weighting is probably wrong. | hypothesistest | derived |
| — | Second- and third-choice packs offered alongside the firstinnovation | Offering the top 3 both hedges model error and generates validation data for free. | inferredADOPT | derived; supports I59 |
| — | TasteProfile schema: per-knob posterior mean + interval + n_observationsinnovation | Makes uncertainty first-class and auditable in the output contract. | hypothesisADOPT | derived |
| — | PreferenceConfidence as a per-knob vector, not a scalarinnovation | Mirrors I55 in the output contract. | hypothesisADOPT | derived |
| — | DesignDNA as a versioned, reproducible artefactinnovation | Pin the model version, catalogue version, and elicitation transcript so a profile is reproducible. | inferredADOPT | token-pack-science pack.version discipline |
| — | Preference vector pre-filters P05 component suggestionsinnovation | The same vector that picks a pack can rank components, so elicitation pays for itself twice. | observedtest | live demo states this intent |
| — | Preference vector seeds image-generation roundsinnovation | Reuse for generated imagery. | observedtest | live demo states this intent |
| — | Per-context confidence (marketing page vs dense dashboard)innovation | A client's density preference on a landing page may differ from a dashboard. Model context as a conditioning variable rather than assuming one profile fits all surfaces. | hypothesistest — this is the open question with the worst cost/benefit | derived; parts.json open question |
| — | Single profile with per-context OFFSETS rather than separate profilesinnovation | A cheaper resolution of I78: one taste vector plus small learned context deltas. | hypothesistest — preferred form of I78 | derived |
| — | Stated vs revealed preference agreement metricinnovation | Measure whether what the client PICKS matches what they later APPROVE in delivered work. The single most commercially important validation. | observedADOPT — long-horizon | Pinterest deprecated onboarding interests in favour of engagement clusters |
| — | Fall back to a safe default pack on low confidenceinnovation | If confidence is too low, ship an industry-appropriate default rather than a badly-inferred pack. | inferredADOPT | Shopify SDUI unknown-layout fallback; Netflix popular-set fallback |
| — | Log spec/pack collisions as a coverage signalinnovation | If two different clients converge to identical packs, that tells you which knob to add. | observedADOPT | ui-pick-to-spec §6 failure mode 4 |
| — | Human review gate before pack is committed to a buildinnovation | A designer confirms the selected pack before it drives a client deliverable. | hypothesisADOPT for v1, revisit | derived |
| — | Profile portability across a client's multiple projectsinnovation | Reuse the DesignDNA for the same client's later work. | hypothesistest | derived |
| — | Explicit re-elicitation rather than silent drift trackinginnovation | Both major platforms found stated onboarding preference decays against behaviour. Prefer a cheap explicit re-run over an invisible decay model. | inferredADOPT | Netflix supersession; Pinterest UIC replacement |
| — | Revealed-preference signal from delivered-work approvalsinnovation | Client approvals/rejections of delivered designs are the highest-quality preference signal available and cost nothing to collect. | inferredADOPT — long-horizon | Hinge infers rankings from like/pass; Stitch Fix updates in real time |
| — | Regulated industries constrain the stimulus space a prioriinnovation | Healthcare, legal, insurance, mortgage clients should not be shown high-chroma, high-motion, low-contrast packs at all — prune before eliciting. | inferredADOPT | b2b-template-shelf-report.md: healthcare PHI, law firm privilege, insurance document authority |
| — | Accessibility floor is not a preference axisinnovation | Contrast below AA is never offered regardless of stated preference. The client cannot choose to fail WCAG. | observedADOPT | token-pack-science Gates C and D |
| — | End-user vs operator preference divergenceinnovation | For the 6/17 industries where `portal` is secondary, the person choosing (the operator) is not the person using (the end customer). Elicit from the operator but weight toward end-user norms. | inferredADOPT — open design problem | b2b-template-shelf-report.md: property_management tenant portal is 'a distinct untrusted identity'; it_services_msps 'per-client tenancy is the product' |
| — | Density preference conditioned on archetypeinnovation | Two archetypes carry the majority of catalogue demand; density expectations differ sharply between a dense case-workflow console and a sparse client portal. | inferredADOPT — cheapest useful form of I78 | b2b-template-shelf: case_workflow primary for 6/17, portal secondary for 6/17 |
| — | Conservative-industry prior on chroma and motioninnovation | Seed these industries from a low-chroma, low-motion region of knob space. | inferredADOPT | accounting_firms, law_firms, insurance_agencies, mortgage_brokers variant deltas |
| — | Expressive-industry priorinnovation | Seed from a higher-chroma, higher-expressiveness region. | inferredADOPT | course_creators, marketing_social_media_agencies, real_estate variant deltas |
| — | Industry prior must be overridable within 3 roundsinnovation | Guarantee the prior is a starting point, not a cage: a client must be able to move out of their industry's region quickly. | hypothesisADOPT | derived |
| — | Bespoke motion and illustration explicitly out of scopeinnovation | The spec bottleneck loses hand-tuned parallax and custom artwork. Do not let a knob imply a capability that does not exist. | observedADOPT | ui-pick-to-spec §6 failure modes 1 and 2 |
| — | The preference learner must itself be A/B tested against a static defaultinnovation | Without a holdout comparing elicited packs to an industry-default pack, we cannot claim the elicitation works at all. | observedADOPT — the single most important methodological gate | Spotify Engineering: a bandit is 'a feature you've built, not an experimental method' |
| — | Measure the knob-correlation matrix before assuming low effective dimensioninnovation | The whole few-rounds argument rests on effective d < 7. Measure it, do not assume it. | hypothesisADOPT — falsifier F3 | derived §4; Jamieson & Nowak's d log n is contingent on the embedding assumption holding |
| — | Do not cite Miller 7+/-2 to justify gallery sizeinnovation | Citing it would not survive a technical client's scrutiny. | observedADOPT as a writing rule | Miller 1956 is about absolute judgment and memory span, not interface option counts |
| — | Do not cite the jam study as a design principleinnovation | Choice overload does not robustly replicate; size galleries on information grounds. | observedADOPT as a writing rule | Scheibehenne et al. 2010: D = 0.02, CI95 [-0.09, 0.12], 63 conditions, N = 5,036 |
| — | Do not promise a specific accuracy number before measuringinnovation | No published benchmark measures our task; any quoted figure would be extrapolation. | observedADOPT as a writing rule | ui-pick-to-spec §4.2 no-figure-found; the 728-vs-113 connector error class |
| — | Treat the live demo's convergence claims as unmeasuredinnovation | Our own prototype's marketing claims must not be recycled as evidence into a client deliverable. | observedADOPT as a writing rule | actionist-taste.pages.dev asserts '~10 picks instead of ~50' with no measurement |
| 1 | Continuous preference vector internally, closed pack at the boundaryinnovation | Catalogue can grow without re-eliciting; shipped artefact is always gate-passing. | inferredADOPT — primary recommendation | derived §8; token-pack-science gates A-J; lane synthesis 'learn a vector not a pack ID' |
| 2 | The preference learner must itself be A/B tested against a static defaultinnovation | Without a holdout comparing elicited packs to an industry-default pack, we cannot claim the elicitation works at all. | observedADOPT — the single most important methodological gate | Spotify Engineering: a bandit is 'a feature you've built, not an experimental method' |
| 3 | Warm-start prior from a population aesthetic modelinnovation | Learn the client's DEVIATION from a population baseline rather than their taste from scratch. The largest single efficiency lever reported anywhere in the applied literature. | observedADOPT | LAION aesthetic predictor; Brochu et al. 2010 cut iterations 11.25 -> 6.5 with a learned prior |
| 4 | Perturbation-tolerance testinnovation | Show the converged pack and a +/-1-level perturbation. If clients reliably distinguish and reject, the tolerance argument in §2 collapses and round counts triple. | hypothesisADOPT — falsifier F1, run FIRST | derived |
| 5 | Brand-constraint hard cutoffs before elicitationinnovation | Existing brand colours/fonts become hard prunes of the stimulus space, not soft preferences; all later stimuli satisfy them. | observedADOPT | Sawtooth ACBC must-have/unacceptable mechanism (vendor manual) |
| 6 | Simulate-to-size: Monte Carlo power analysis for round countinnovation | Answer 'how many rounds' EMPIRICALLY by simulating synthetic clients under a BT likelihood rather than arguing from bounds. | observedADOPT — this is how the N question should actually be settled | skpr (R) |
| 7 | Two-week test-retest stability as the primary validation metricinnovation | A model that fits the clicks but changes answer next Tuesday is worthless. Measures whether the thing we claim to measure exists. | inferredADOPT — this is falsifier F7 | Slovic 1995 construction of preference |
| 8 | P05 corpus components as re-themable controlled stimuliinnovation | Use the 8,515-identity corpus as the stimulus pool: identical structure, varying token pack, re-theming is a find-and-replace on ~30 custom properties. | observedADOPT | research/21st-corpus-audit-2026-08-27.md — 86.7% of colour-bearing CSS rules already resolve through var(--token); 87.3% carry a shadcn oklch :root block |
| 9 | 4-up gallery with explicit outside optioninnovation | 2.32 noiseless bits/round; ~0.91 effective at p=0.15; balances information against deliberation cost and avoids the hostility of 'worst'. | inferredADOPT — primary recommendation | derived §3 + Sawtooth 4-5 per screen + live demo |
| 10 | End-user vs operator preference divergenceinnovation | For the 6/17 industries where `portal` is secondary, the person choosing (the operator) is not the person using (the end customer). Elicit from the operator but weight toward end-user norms. | inferredADOPT — open design problem | b2b-template-shelf-report.md: property_management tenant portal is 'a distinct untrusted identity'; it_services_msps 'per-client tenancy is the product' |
26 mined records. Search and inspect the full agent output; nothing is collapsed into a top-ten headline.
| Rank | Identity | What the agent found | Evidence | Source |
|---|---|---|---|---|
| — | P06-SR-001— | Local prior establishing pick-as-query-not-asset, a closed semantic section vocabulary of ~6-12 enum axes per section type, SDUI prior art (Airbnb/Lyft/Spotify/Shopify/DoorDash), a four-layer validation gate, and the requirement that previews be re-rendered from spec rather than shown as original thumbnails. | observedfoundational_input | research/ui-pick-to-spec-2026-08-27.md |
| — | P06-SR-002— | Local prior establishing the closed token-pack gallery thesis, DTCG 2025.10 as interchange format, Radix 12-step role scales with APCA targets, M3 HCT seed-to-role determinism, a complete pack schema and machine gates A-J. | observedfoundational_input | research/token-pack-science-2026-08-27.md |
| — | P06-SR-003— | Names the seven mechanical preference knobs (contrast, radius, shadow, border, typography, density, chroma), identifies Bradley-Terry/Luce as relevant models, and records that forced choice without a 'none' option pollutes the preference model. | observedfoundational_input | research/actionmodel-builder-research-2026-08-26/phase-8/external-opus-inputs/CLAUDE-LANES-SYNTHESIS.md |
| — | P06-SR-004— | Defines P06 ownership: preference axes and stimulus generation, experiment policy, outside option and re-roll behaviour, client design DNA and confidence, preference update over time. Open questions include whether the minimum is 7, 14 or another number of choices. | observedscope_definition | site/system-map/data/parts.json (P06 entry) |
| — | P06-SR-005— | Bradley-Terry gives Pr(i>j) = p_i/(p_i+p_j) with positive real scores; logit form logit Pr(i>j) = beta_i - beta_j. Original citation Bradley & Terry (1952) 'Rank Analysis of Incomplete Block Designs: I. The Method of Paired Comparisons', Biometrika 39(3/4):324-345, doi:10.2307/2334029. Previously studied by Zermelo (1929). Plackett-Luce generalizes to full rankings and satisfies Luce's choice axiom. | observedmodel_foundation | Source ↗ |
| — | P06-SR-006— | Crowd-BT (Chen, Bennett, Collins-Thompson, Horvitz, WSDM 2013, doi:10.1145/2433396.2433420) reduces the number of comparisons required by modelling per-judge reliability and screening unreliable raters; outperformed plain Bradley-Terry and TrueSkill in a 624-judge document-difficulty task. | observeddirectly_relevant_extension | Source ↗ |
| — | P06-SR-007— | Best-worst scaling: with four items, naming best and worst reveals 5 of the 6 implied pairwise relations (only the middle pair stays unknown); with five items, 7 of 10. Invented by Jordan Louviere in 1987; concept traces to A.A.J. Marley; definitive text Louviere, Flynn & Marley (2015, Cambridge University Press). | observedformat_evidence | Source ↗ |
| — | P06-SR-008— | MaxDiff is a strict subset of BWS assuming respondents evaluate all n(n-1) ordered pairs and pick the maximum-difference pair; the source reports that across 14 years of presentations the authors virtually never found a practitioner who admitted using that strategy, with most describing sequential best-then-worst strategies. | observedcaution_on_model_assumptions | Source ↗ |
| — | P06-SR-009— | BWS Case 2 (profile case) has proven usable by populations that struggle with conventional discrete choice experiments; one cited study found nearly all older respondents produced usable BWS data while only about half did for the DCE. | observedaccessibility_evidence | Source ↗ |
| — | P06-SR-010— | shadcn theming exposes semantic token pairs (background/foreground, card, popover, primary, secondary, muted, accent, destructive), structural tokens (border, input, ring), chart-1..5, a sidebar token group, and a radius scale. Surface tokens pair with a matching -foreground token for text and icons. | observedconfirms_local_corpus_finding | Source ↗ |
| — | P06-SR-011— | The 21st bundle corpus carries a standard shadcn :root oklch token block in 87.3% of sampled bundles (n=200) and 86.7% of colour-bearing CSS rules already resolve through var(--token) (n=25), making bundles re-themable as controlled stimuli by swapping ~30 custom properties. | observedstimulus_supply_evidence | research/21st-corpus-audit-2026-08-27.md |
| — | P06-SR-012— | Information-theoretic derivation: 7 knobs x 4 discriminable levels = 16,384 profiles = 14 bits for exact identification; selecting among a closed 30-pack gallery is only ~4.9 bits, cutting the elicitation budget by roughly two thirds. | inferredsuperseded_by_SR_015 | lanes/S1-L3/owner-finding-p06-choice-budget.md |
| — | P06-SR-013— | Live taste-picker prototype INSPECTED. Four rendered interface cards per round ('every card is a bundle of five knobs'), explicit outside option 'none of these' plus 'start over', ~10 picks total, later rounds target the 'least-resolved knob' holding other inferred preferences steady. Five of seven knobs varied (palette, radius, type, density, shadow). Live belief panel 'What the system believes about you'. Estimator is win-rate counting with Laplace smoothing, a zeroth-order Bradley-Terry; the page names choix or a GP preference model as production intent. | observedRESOLVED — adopt the interaction model, replace the estimator, add a real stopping rule | Source ↗ |
| — | P06-SR-014— | Commercial denominator BUILT: 103 surfaces across 10 categories in top-companies.jsonl, 10 ranked. | observedPARTIALLY RESOLVED — enumerated, thinly verified | Commercial preference-elicitation denominator (Stitch Fix, Netflix cold-start, Spotify onboarding, Pinterest interest picker, Sawtooth, Conjointly, Qualtrics, Canva Brand Kit, Figma First Draft) |
| — | P06-SR-015— | Channel-capacity derivation: 7 knobs at plausible level counts give 36,000 packs = 15.14 bits; resolving each knob to +/-1 level leaves 11.09 bits deliberately unresolved, so only 4.04 bits must be acquired. Through a binary symmetric channel at p=0.15 a 4-up-plus-outside-option screen delivers ~0.91 effective bits, giving ~5 rounds; recommend 8-12 for myopic acquisition, exploration overhead and knob correlation. | observedprimary_derivation | first-principles.md §1-§3 (derivation, this run) |
| — | P06-SR-016— | Closest published prior art. Verbatim: 'The mean iteration count necessary for finding satisfactory results was 5.36 with SD = 2.69'; 'One plane-search subtask took 14.8 seconds on average'. Participants continued to 15 iterations regardless; 5 of 6 pressed satisfaction within that. Synthetic functions at 5D, 15D, 10D, 20D. Grid was 5x5 for photo enhancement but reduced to 3x3 on a 13-inch display for body-shape design. | observedprimary_empirical_anchor | Source ↗ |
| — | P06-SR-017— | Theoretical licence for few comparisons over 7 knobs. Verbatim: objects in d-dimensional Euclidean space ranked by distance from a reference, 'the number of possible rankings grows like n^{2d}', and an algorithm identifies a randomly selected ranking using 'just slightly more than d log n adaptively selected pairwise comparisons, on average'; if comparisons are chosen at random 'almost all pairwise comparisons must be made'. | observedtheoretical_licence | Source ↗ |
| — | P06-SR-018— | The counterweight that prevents over-promising. An adaptive counting algorithm with confidence-interval stopping recovers the ranking 'optimal up to logarithmic factors' with no structural assumption on the pairwise probability matrix; and a lower bound proves parametric choices (Bradley-Terry, Thurstone) 'offer at most logarithmic gains for stochastic comparisons'. | observedconstrains_optimism | Source ↗ |
| — | P06-SR-019— | The only hard question-count formula found in commercial practice, quoted verbatim: 'we recommend displaying either four or five items at a time (per set or question)'; never more than half the study's item count per set; ask enough sets 'such that each item has the opportunity to appear from three to five times per respondent'; formula '3K/k' where K = total items, k = items per set. Beyond ~5 items per set 'The gains in precision of the estimates are minimal.' Assumes HB estimation. | observedformat_corroboration | Source ↗ |
| — | P06-SR-020— | A SHIPPED adaptive visual-preference elicitor. Uses 'the styles you pick (and the ones you don't!)' to build a reusable --sref code. Each regenerated preview set is one refinement round. Documented convergence: 'Most styles stabilize after 5-10 rounds', rounds 10-15 add detail, 'Past round 15, changes are small and subtle'. Skipping is available but explicitly NON-informative: 'skipping does not affect your style development'. User-terminated via End Session. | observedclosest_shipped_analogue | Source ↗ |
| — | P06-SR-021— | The most important negative result in the commercial sweep. Pinterest REPLACED onboarding followed-interests as a retrieval condition with user interest clusters derived from actual engagement, because the onboarding signal 'skewed heavily toward dominant interests' and was 'static, not evolving with behavior'. | observedcounter_evidence_on_drift | Source ↗ |
| — | P06-SR-022— | Vendor-documented supersession rule. Title picker at profile creation is explicitly OPTIONAL; skipping falls back to 'a diverse and popular set of titles'. Once engagement begins it 'supersedes' the initial preferences, with recent activity outweighing older. | observeddrift_precedent | Source ↗ |
| — | P06-SR-023— | Settles a question that would otherwise silently shape the UI. Meta-analysis of 63 conditions from 50 experiments (N=5,036): mean effect size D = 0.02, CI95 [-0.09, 0.12]; trimmed D = 0.001. 'No sufficient conditions could be identified' and 'adverse consequences due to having too much choice are not a robust phenomenon'. | observedfrees_gallery_sizing | Source ↗ |
| — | P06-SR-024— | Two writing rules. Miller's 7+/-2 concerns absolute judgment of unidimensional stimuli and immediate memory span for chunks, NOT interface option counts; Miller himself treats the recurrence of ~7 with irony. Slovic establishes that preferences are frequently CONSTRUCTED during elicitation rather than retrieved, which bounds how much precision is even meaningful and argues for test-retest stability as the validation metric. | observedepistemic_guardrails | Source ↗ |
| — | P06-SR-025— | Licence trap of exactly the class this project was warned about. The repo (803 stars) shows a BSD-3-Clause badge, but the LICENSE body reads: 'Microsoft permits only Xbox Live games or non-commercial projects to use TrueSkill(TM). If your project is commercial, you should find another rating system.' GitHub's API reports only NOASSERTION. Conversely huawei-noah/HEBO shows a bare NONE badge but carries a real MIT licence one directory down at HEBO/LICENSE. | observedDISQUALIFIES sublee/trueskill for paid client work; openskill.py (MIT) is the substitute | Source ↗ |
| — | P06-SR-026— | Methodological gate. With a contextual bandit there is no single 'best button', only a 'best system', so it still requires an experiment against the static original — 'the bandit is a feature you've built, not an experimental method'. Optimizely's docs corroborate the mechanism: no statistical significance is calculated for contextual-bandit optimisations. | observedmakes_Gate_6_non_negotiable | Source ↗ |
Source: site/system-map/data/parts.json#P06 · preserved research datasets remain inline below their source-backed synthesis.