Why Your Coding Agent Skills Are Probably Broken (And How to Fix Them)

I used to think that bolting more skills onto my coding agent would automatically make it smarter—until I realized most of the 47,000+ skills floating across 6,300 repositories are untested, AI-generated, and held together by informal 'vibe-checks' that would never pass muster in a real code review. The uncomfortable truth is that the agent skill ecosystem is shipping behavioral triggers without unit tests, and that gap is silently costing you tokens, reliability, and trust. After diving deep into the research, design patterns, and hard-won lessons from production systems, I've come to see skill design as a genuine engineering discipline—not a prompt-writing exercise. In this article, I'll walk you through the seven proven design patterns, the evaluation frameworks that turned a 66.7% pass rate into 100%, and the architectural principles that separate agents that scale from agents that stall.

The Skill Landscape: 47,000 Skills and Most Are Untested

Agent skills are fundamentally modular folders that bundle instructions, scripts, and resources to extend what an agent can do, all without forcing a full model retraining cycle. When I look at the SkillsBench dataset, the scale is staggering: the community has produced over 47,000 unique skills scattered across more than 6,300 repositories. Yet when I dig into the metadata, most of these skills have never seen a structured test suite. They are often generated by AI, committed, and then left to sit with no verification pipeline, no regression checks, and no clear ownership.

Why "Vibe-Checks" Are Not Validation

I see teams repeatedly fall into the same trap. They install a skill, run it manually against two or three prompts, decide it "feels right," and ship it to production. This informal "vibe-check" approach is functionally identical to shipping application code without unit tests. The danger here is that skills do not fail loudly; they act as behavioral triggers that can misfire silently. In production, a malformed instruction or an outdated API reference does not throw a neat stack trace. It simply produces subtly wrong behavior that propagates downstream, corrupting outputs and eroding trust in the agent's decisions.

Two Categories, One Blind Spot

Not all skills serve the same purpose, and I think the testing strategy needs to reflect that. There are two fundamental types:

• Capability skills bridge gaps where the base model cannot consistently perform a task on its own. Evaluations here serve a dual purpose: they verify the skill works today, and they reveal when underlying model improvements have made the skill redundant. Without tests, you end up maintaining scaffolding that the base model no longer needs.
• Preference skills encode specific team workflows, style guides, or organizational standards. For these, evaluations act as compliance checks that confirm the agent adheres to internal conventions rather than drifting toward generic defaults.

Both categories share the same vulnerability. Without automated validation, you cannot distinguish between a skill that genuinely extends capability and one that is dead weight—or worse, actively harmful.

What Rigorous Evaluation Actually Looks Like

The Interactions API skill case study gives us a concrete blueprint for why systematic testing beats intuition. Before evaluation, the skill was passing only 66.7% of the time. After targeted fixes driven by structured testing, that number climbed to 100%.

The improvements were granular and measurable. Evaluation identified and eliminated token waste, which directly reduces inference cost and latency. It caught usage of deprecated APIs before those calls could reach production and trigger errors. Finally, it enforced consistent SDK adoption, ensuring that every generated code snippet followed the same library patterns rather than fragmenting across conflicting versions.

When I look at the broader ecosystem of 47,000 skills, the lesson is obvious. Modular architecture is worthless if the modules themselves are unverified. If your skill pipeline lacks automated evaluation, you are not extending your agent's capabilities—you are expanding its attack surface.

Seven Design Patterns for Agentic Skills

When I look at how modern coding agents actually handle skill orchestration, it becomes clear that the implementation strategy matters far more than the prompt engineering behind it. Jiang et al. (2026) systematize this space into seven distinct design patterns, and each one reveals a specific architectural bet about efficiency, safety, and composability.

Packaging and Execution Fundamentals

The most common approach is P1: Metadata-Driven Progressive Disclosure, where agents discover capabilities through compact metadata summaries—names, descriptions, and trigger conditions—while deferring the full instruction payload until selection. I see this in tools like Claude Code and Semantic Kernel. It is ruthlessly token-efficient, but the attack surface shifts upward: if an adversary poisons the metadata layer, the agent loads malicious instructions without ever inspecting the payload at discovery time.

Then there is P2: Code-as-Skill, which treats skills as executable scripts—Python functions or shell commands—that are deterministic, testable, and composable. Voyager, CodeAct, and SWE-agent all lean on this model. I find this pattern attractive because it gives me actual governance over execution paths, though I have to watch for code injection and API brittleness when underlying interfaces change.

Sitting between these extremes is P5: Hybrid NL+Code Macros, combining natural-language specifications with executable components. Claude skills and ReAct prompts use this to stay human-auditable while remaining machine-executable. The risk here is boundary ambiguity—when the natural-language instruction contradicts the encoded logic, the agent has no clear arbitration rule.

Control, Evolution, and Quality Assurance

P3: Workflow Enforcement takes a different tack by hard-gating agent behavior through rigid process constraints. Tools like LATS and TDD agents sacrifice flexibility for reliability. I notice that while this raises the barrier for casual errors, prompt injection can still bypass rule sets if the gate logic itself is manipulated through the context window.

P4: Self-Evolving Skill Libraries attempt to solve the maintenance burden by coupling execution with automated quality assessment. Voyager and CRADLE explore this, but the data gives me pause. SkillsBench evaluations show that self-generated skills actually average -1.3 percentage points compared to human-curated equivalents, which tells me that quality control—not generation—is the real bottleneck in autonomous skill evolution.

Abstraction and Distribution Risks

Moving up the stack, P6: Meta-Skills treat skill creation itself as a skill. Self-Instruct and various skill generators fall here. I see the elegance, but I also see the danger: recursive error amplification means a single flawed meta-skill can spawn an entire family of broken descendants downstream.

Finally, P7: Plugin/Marketplace Distribution enables community-driven scale through versioned packages with dependency metadata, as seen in the OpenAI GPT Store and MCP servers. The composability is high, but the ClawHavoc study found a 36.8% malicious rate in marketplace plugins, which makes supply-chain attacks the dominant threat model here.

If I map these against the trade-off matrix, the engineering compromises become concrete. P1 keeps context cost low but pays for it with low determinism and only medium composability and governance. P2 is the standout: low context cost, high determinism, high composability, and high governance. P3 raises context cost to medium while keeping determinism and governance high, though composability stays medium. P4 sits at medium across context cost, determinism, and composability, but drops to low governance—a red flag for me. P5 is uniformly medium, which reads as safe but uninspiring. P6 is the most expensive, with high context cost and low determinism, composability, and governance. P7 offers low context cost and high composability, but determinism varies and governance only reaches medium-high despite that 36.8% malicious rate. Choosing a pattern is not about picking the newest abstraction; it is about deciding which failure mode you are actually prepared to mitigate.

Single Agent + Tools Beats Multi-Agent for Coding

When I look at how most teams architect their coding agents, I see a recurring mistake: they reach for multi-agent orchestration when the data clearly points in the opposite direction. Anthropic's 2025 research on OpenCode agent creation shows that single agent + tools consistently outperforms multi-agent setups for coding workloads. The reason is straightforward—most coding tasks simply don't parallelize the way research tasks do. While multi-agent systems deliver a 90.2% improvement in research scenarios because independent search operations don't collide, code is fundamentally sequential. Sub-agents cannot cleanly coordinate edits to the same file without stepping on each other, and the overhead of merge logic often erases any theoretical speed gain.

Why Multi-Agent Systems Fail at Code

The architectural mismatch becomes obvious when I compare task independence. Research benefits from parallel search because each agent can scour a different corner of the internet without conflict. Code, however, requires linear reasoning. When multiple agents attempt to modify shared functions, imports, or class definitions simultaneously, the coordination cost explodes. I noticed that coding pipelines involve far fewer parallelizable tasks than people assume, which is why the single-agent model—armed with a robust tool suite—keeps the context unified and the edit history coherent.

Designing Prompts at the "Right Altitude"

One of the most effective principles I've seen is keeping system prompts lean. Anthropic's benchmark suggests the sweet spot sits around ~500 tokens, built around clear heuristics rather than exhaustive rule lists. In my analysis, this works because examples outperform edge-case documentation. Showing one canonical example teaches the agent the pattern far more effectively than listing twenty hypothetical scenarios. When I strip a prompt down to this "right altitude," the agent stops drowning in conditional logic and starts executing with sharper intent.

Just-in-Time Context Loading

Pre-loading an entire codebase into the context window is a trap I see teams fall into repeatedly. The alternative is just-in-time context loading, where the agent discovers information layer by layer through tool calls. I find that guiding behavior with file metadata—size, name, and timestamps—lets the agent pull exactly what it needs instead of guessing. Dumping the full repository into the prompt causes the model to waste tokens on irrelevant files, effectively drowning it in noise before it writes a single line of code.

The CLAUDE.md Pattern

To eliminate repetitive setup, the CLAUDE.md pattern creates a project context file that auto-loads at the start of every session. When I implement this, I populate it with bash commands, code style guidelines, common file patterns, and workflow rules. Because the file lives in the repository and syncs through git, the entire team shares the same baseline context without manually pasting instructions into each chat. It removes friction and prevents the "what are our linting rules again?" loop.

Where Performance Variance Actually Comes From

If you're optimizing a coding agent, the numbers are unambiguous. Token usage explains 80% of performance variance, while the number of tool calls and model choice each account for only around 10%. This tells me that architectural decisions around context management and prompt length matter far more than swapping models or adding agent count. Before you spin up another sub-agent, audit your token budget—you're likely bleeding performance there.

The Code Intelligence Layer: Five Attributes Your Agent Actually Needs

I’ve noticed that most LLM coding agents are still architected like traditional IDE plugins, and that mismatch is exactly where reliability collapses. IDEs tolerate human-paced, incremental queries and visual feedback loops. Agents consume context programmatically and pay for every token. The intelligence layer must be rebuilt around five attributes that prioritize machine consumption over human browsing.

One Call Should Carry the Full Context

The first attribute is aggressive reduction of tool calls. When an agent triggers a rename operation, it cannot afford the typical language-server dance of requesting locations, then snippets, then change summaries. I look for a single invocation that returns everything required to act: the exact usage sites, relevant bounded code snippets, and a summary of mechanical changes needed. This lets the agent plan and apply the fix in one step, which directly cuts latency and preserves precious context-window space.

Token-Efficient Outputs and Stable References

Second, the output format itself needs to change. I see too many tools dumping exhaustive JSON structures or massive location lists that mirror internal language-server representations. That is pure noise. What an agent actually needs resembles what I would ask a senior teammate for: concise signatures, bounded code snippets, and dense summaries. The response must be information-rich but structurally tight.

That efficiency only works if the third attribute—stable references—is present. Multi-step workflows like rename followed by fallout fixes depend on identifiers that survive file modifications. I need persistent symbol keys or qualified names that remain valid across tool calls. Without this stability, the agent is forced to re-query locations after every edit, which destroys the very efficiency the layer is supposed to provide.

First-Class Diagnostics and Big-Repo Navigation

Fourth, the compilation and diagnostics loop cannot be an afterthought. I want diagnostics exposed as a primary signal, not buried behind opaque secondary calls. When the agent introduces a regression, it should detect the error, apply an automatic code action, and confirm resolution within the same workflow. Treating diagnostics as a first-class citizen removes the guesswork.

Finally, none of this matters if the agent cannot navigate large repositories. Real codebases demand:

• Call graphs that trace execution flow
• Type hierarchies for inheritance analysis
• Impact analysis before sweeping changes
• Precise cross-reference searches instead of naive text matching

When these are missing, I watch agents fall back to grep, which is disastrous. A naive text search for a common symbol returns thousands of irrelevant hits inside comments, strings, and Javadocs. Sifting through that burns thousands of tokens and buries the actual signal in noise. The result is higher latency, inflated costs, and brittle reasoning.

These five attributes are not optional upgrades. They form the minimum viable foundation for an agent that can operate reliably outside of toy examples.

Skill Security: Sandboxing, Script Approval, and Supply Chain Risks

When I look at how agent skills handle script execution, I see two fundamentally different attack surfaces that most implementations fail to separate properly. File-based scripts are auto-discovered from .py, .js, .sh, .ps1, .cs, and .csx files inside the scripts/ subdirectory by default. These expect arguments as a JSON array of strings, where each element maps directly to a positional command-line argument, and they require an explicit script runner—something like SubprocessScriptRunner.RunAsync—to actually execute. On the other hand, code-defined scripts register as in-process delegates, bypass the script runner entirely, and automatically convert their parameters to JSON Schema for agent argument passing. That distinction matters because the security model for an out-of-process subprocess is completely different from an in-process delegate with direct memory access.

Execution Models and Their Security Boundaries

• File-based scripts: Because these invoke external processes, they inherit all the risks of shell execution—path injection, argument parsing vulnerabilities, and environment leakage. The runner is the only gatekeeper.
• Code-defined scripts: Running in-process means no process boundary, but also means a compromised delegate has immediate access to the agent's memory space and any injected services.

I always treat SubprocessScriptRunner as a development convenience, not a production boundary. In production, you need real isolation.

Sandboxing and Resource Containment

If you're executing arbitrary skill scripts, running them inside isolated environments is non-negotiable. I recommend containers, seccomp profiles, or tools like firejail to restrict filesystem and network access. Beyond isolation, enforce strict resource ceilings:

• CPU throttling to prevent cryptomining or algorithmic denial-of-service.
• Memory caps to stop runaway allocations from crashing the host process.
• Wall-clock timeouts so a hanging script doesn't block the agent indefinitely.

Input validation needs to be aggressive: maintain an explicit allow-list of executable scripts rather than trusting directory contents, and validate every argument against expected schemas before it reaches the runner.

Audit Trails and Human-in-the-Loop Approval

Structured logging isn't optional here. I implement audit trails covering three specific phases: skill loading, resource reading, and script execution. Without this, you can't reconstruct what happened during a breach.

For high-risk operations, gate execution behind human approval. Set ScriptApproval = true or require_script_approval=True so that scripts don't run autonomously until explicitly authorized. This adds friction, but it's the difference between an automated attack and a failed attempt waiting for human review.

Production Hardening and Supply Chain Reality

Moving to production means replacing demonstration runners with proper dependency injection for service access and customizing script discovery through AgentFileSkillsSourceOptions rather than accepting default directory scans. Hard-coded secrets in skill content remain a persistent vulnerability—never embed credentials inside script files or skill manifests.

The supply-chain risk is quantifiable. The ClawHavoc study found a 36.8% malicious rate in plugin and marketplace distributions, which directly exposes P7 pattern skills to compromised dependencies. When I see that statistic, I stop treating marketplace plugins as trusted code and start treating them as untrusted inputs requiring the same sandboxing and approval workflows as user-authored scripts.

Effective Skill Design Principles and the Skill Factory Pattern

When I look at how coding agents fail in production, the root cause is rarely the underlying model. More often, the skill itself is overloaded, ambiguous, or untested against its own description. The design principles for agent skills are straightforward, but skipping any one of them creates compounding reliability issues.

Core Design Principles for Reliable Skills

The first rule I always check is single-purpose focus. Each skill should handle one specific job. When a skill tries to juggle multiple responsibilities, the agent struggles to determine intent, and reliability drops immediately.

I also prefer instructions over scripts whenever possible. Scripts only make sense when I need deterministic behavior or external tooling integration; otherwise, they add unnecessary complexity and extra failure points that are painful to debug later.

For workflow clarity, I write imperative steps with explicit inputs and outputs. Agents follow these directives more reliably than vague descriptions. But writing clear steps isn't enough. I test prompts against the skill description to verify trigger behavior—specifically, I confirm the description accurately captures when the skill should activate and, just as importantly, when it should stay dormant. A misaligned description is one of the fastest ways to get erratic agent behavior.

To keep things maintainable, I follow progressive disclosure:

• SKILL.md stays under 500 lines
• Detailed references move to separate files
• This prevents the main skill file from becoming an unreadable monolith

Packaging, Distribution, and Configuration

When I need to distribute skills beyond a single repository, I bundle them as plugins. These packages can include:

• Multiple skills
• App mappings
• MCP server configurations
• Presentation assets

It's a much cleaner way to share capabilities than copying individual files around.

For local configuration, I use [[skills.config]] inside ~/.codex/config.toml to disable skills without deleting them entirely—just set enabled = false. I also add agents/openai.yaml to configure UI metadata like display name, description, icons, brand color, and the default prompt. This same file lets me declare tool dependencies, such as required MCP servers, keeping the runtime requirements explicit.

The Skill Factory Pattern and Meta-Skill Generation

The Skill Factory pattern is where things get interesting. Instead of manually authoring every capability, I equip ADK agents with meta-skills that autonomously generate new skills. The built-in skill creator operates as an inline skill, and its resources field contains two critical L3 references: the full Agent Skills specification (skill-spec.md) and a working example skill (example-skill.md). These references act as guardrails, ensuring generated outputs match the expected format.

Generated skills must adhere to strict conventions:

• Kebab-case naming with a 64-character maximum
• Descriptions under 1,024 characters
• Clear step-by-step instructions
• Domain-specific details live in references/ files
• The main SKILL.md stays under 500 lines
• Output must be direct and actionable, ready for the user to save without extra cleanup

Even with automated generation, I never skip human-in-the-loop review of the generated SKILL.md. After that, I run post-generation evaluation using ADK's testing framework to validate effectiveness before the skill ever touches a production environment. This two-stage gate—human review plus automated testing—catches structural errors and functional gaps that pure generation often misses.

Cross-Platform Skill Distribution and the Skill Lifecycle

I noticed that Agentver solves cross-platform fragmentation by decoupling skill content from agent-specific deployment logic through a strictly data-driven architecture. The platform currently supports 43+ coding assistants, and the mechanism that makes this scale is a single source of truth: the @agentver/agent-definitions package inside packages/agent-definitions/. Every agent configuration path, file format, and installation location is defined exclusively there. When a coding assistant updates its configuration format or adds a new skill path, maintainers modify exactly one entry in that data layer, while the skill content itself remains completely unchanged. Skills declare their target agents via manifest configuration, and Agentver handles translation and file placement automatically. Adding support for a newly released agent is therefore a pure data change—specifying the skill directory path and file naming convention in agent-definitions—with zero architectural modifications required. The @agentver/cli tool and platform services consume this data layer to detect installed agents on a user's machine and deploy skills to the correct locations, such as .claude/skills/ for Claude Code and .cursor/skills/ for Cursor.

The skill lifecycle model is not a passive pipeline; it is an active loop built around seven explicit stages:

1. Discovery – identifying a new capability or pattern.
2. Practice/Refinement – iterating until the skill is reliable.
3. Distillation – stripping the skill down to its portable essence.
4. Storage – archiving in a retrievable format.
5. Retrieval/Composition – pulling the skill and combining it with others.
6. Execution – running the skill in a live environment.
7. Evaluation/Update – measuring outcomes and versioning accordingly.

What I find particularly well-designed are the explicit feedback loops that prevent skills from becoming stale. Evaluation feeds back into Practice, Retrieval feeds back into Storage, and Execution feeds back into Discovery. This creates a self-correcting system where a skill that underperforms during execution can trigger a new discovery phase, and retrieval patterns can reshape how skills are stored and indexed.

Representation and Scope Taxonomy

To manage this complexity without chaos, Agentver classifies skills using a Representation × Scope Taxonomy. This two-axis matrix is more than a labeling exercise; it determines how the platform translates a skill's intent into agent-specific instructions and which safety constraints apply during execution.

• Representation categories:

• Natural-language – instructions written in plain text.

• Code-as-skill – executable scripts or functions.

• Tool macros – predefined sequences of tool invocations.

• Policy-based – rule-governed behavior constraints.

• Hybrid – combinations of the above.

• Scope categories:

• Single-tool – confined to one utility or agent feature.

• Multi-tool orchestration – coordinating several tools in sequence.

• Web interaction – browser or API-based tasks.

• OS/desktop – file system or operating-system-level operations.

• Software engineering – code generation, refactoring, and testing workflows.

• Robotics/physical – hardware or embodied agent actions.

OpenCode's Dual-Agent Safety Model

On the execution front, OpenCode's dual-agent architecture cleanly separates destructive operations from exploratory analysis. Build Mode is where the agent gets full read/write access, can execute shell commands, install dependencies, and run test suites—essentially the permissions needed to ship real changes. Plan Mode restricts the agent to read-only operations for analysis and planning, eliminating the risk of unintended modifications while still allowing deep code comprehension. Both modes share robust LSP support, which means real-time type information, symbol navigation, reference tracking, and diagnostic understanding are available whether the agent is safely planning or actively building. To me, this shared LSP layer is what makes the split trustworthy: the agent reasons from the same structural understanding of the codebase in both modes, so the transition from planning to execution does not introduce cognitive drift.