[prompt-analysis] Copilot PR Prompt Analysis - 2026-09-03

[github-actions[bot]]

🤖 Copilot PR Prompt Pattern Analysis - 2026-09-03

Summary

Analysis Period: Last 30 days
Total PRs: 1000 | Merged: 844 (84.4%) | Closed: 147 (14.7%) | Open: 9 (0.9%)

Prompt Categories and Success Rates

Category	Total	Merged	Success Rate
Bug Fix	816	691	84.7%
Feature Addition	124	112	90.3%
Security / CVE	29	18	62.1%
Testing	7	6	85.7%
Documentation	3	2	66.7%
Refactoring	2	2	100.0%
Other	19	13	68.4%

Note: PR categorization is dominated by "bug fix" style prompts (81.6% of the sample), since most Copilot tasks in this repo are framed as fixing/resolving an issue. Security/CVE-related prompts were split out as their own bucket given their materially lower success rate.

Prompt Analysis

✅ Successful Prompt Patterns

Common characteristics in merged PRs:

• Average prompt length: 178.0 words
• Most common (non-boilerplate) keywords: workflow, run, actions, copilot, agent, sub/chef/sous (internal codenames), gpt
• Feature-request prompts (add, implement, create, support) had the highest success rate (90.3%) of all categories.
• Refactor and testing prompts, while smaller in sample size, also converted well (100% and 85.7% respectively).

Example successful prompts:

1. PR Add recursive package manifest imports #58233: "Add recursive package manifest imports" → Merged
2. PR Move max AI credits expression to step env and validate integer in shell run block #58116: "Move max AI credits expression to step env and validate integer in shell run block" → Merged
3. PR Fix call_workflow samples replay dropping workflow_name during ingestion #58113: "Fix call_workflow samples replay dropping workflow_name during ingestion" → Merged

❌ Unsuccessful Prompt Patterns

Common characteristics in closed PRs:

• Average prompt length: 206.4 words (~16% longer than merged prompts)
• Frequent terms: issue, cve, report, engine, call — closed PRs skew toward vulnerability/CVE remediation and workflow-engine internals.
• Security/CVE-tagged prompts had the lowest success rate (62.1%) — several were "no code changes" / report-only PRs or WIP container-pin refreshes that were superseded rather than merged.

Example unsuccessful prompts:

1. PR No-op: exploitation-error grader already implemented on main #58035: "No-op: exploitation-error grader already implemented on main" → Closed
2. PR Suppress runner-guard alert for Playwright loopback healthcheck #57859: "Suppress runner-guard alert for Playwright loopback healthcheck" → Closed
3. PR Add Claude Code LSP support #57258: "Add Claude Code LSP support" → Closed

Key Insights

• Pattern 1: Feature-style prompts ("add X", "implement Y") merge at 90.3%, notably higher than the overall 84.4% average — concrete, additive asks with a clear scope tend to land cleanly.
• Pattern 2: Security/CVE-driven prompts merge at only 62.1%, the weakest category. Several were investigative/no-op tasks ("scan report", "no code changes") rather than concrete fixes, suggesting CVE remediation prompts benefit from an explicit, scoped fix instruction rather than an open-ended "investigate/report" framing.
• Pattern 3: Closed PRs average ~28 more words than merged ones (206 vs 178). Overly long prompts may indicate either genuinely complex/ambiguous asks or scope creep — conciseness combined with a single clear action verb correlates with higher merge rates.

Recommendations

1. DO: Frame prompts around a single, concrete action verb ("add", "fix", "implement") with a specific target (file, function, workflow step) — this is the pattern behind the highest success rates (feature: 90.3%, refactor: 100%).
2. DO: For CVE/security work, prefer prompts that request an explicit code change/patch, not just "review", "report", or "assess" — investigative prompts in this sample skewed toward closure.
3. AVOID: Long, multi-part prompts bundling several unrelated asks; the closed-PR set averaged noticeably more words than the merged set, hinting at scope or clarity issues.

Historical Trends

Date	PRs	Merge Rate	Top Category
2026-09-03	1000	84.4%	Bug Fix
2026-09-02	1000	83.9%	Bug Fix
2026-09-01	1000	84.1%	Bug Fix
2026-08-31	994	84.3%	Bug Fix

Trend: Merge rate has held steady in a narrow 83.9%–84.4% band over the past 4 days of tracked data — no material shift in overall Copilot PR success rate. Bug-fix prompts remain the dominant category by volume every day, consistent with this repo's issue-driven workflow.

---

Generated by Copilot PR Prompt Analysis (Run: 33803610906)

Generated by 🔍 Copilot PR Prompt Pattern Analysis · copilot · auto · 22.9 AIC · ⌖ 10 AIC · ⊞ 11K · ◷

• expires on Sep 4, 2026, 12:48 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Copilot PR Prompt Pattern Analysis.

A newer discussion is available at Discussion #58606.