5 posts tagged with "llm"

Meta description: Vercel released an open CLI that lets you install reusable "skills" into any AI coding agent — Claude Code, Cursor, Copilot, and 40+ more. Here's why it matters and how to use it.

The Problem: Every Agent Starts From Zero​

If you've used more than one AI coding agent — Claude Code, Cursor, Copilot, Windsurf, Codex — you've probably noticed something annoying: each one needs its own setup. Your carefully crafted instructions, tool configurations, and workflow rules don't transfer. Switch agents, start over.

And if you're a team? Multiply that pain. Every developer configures their own agent differently. Consistency is a pipe dream.

This is the problem Vercel's skills CLI was built to solve.

What Is Vercel Skills?​

Vercel Labs released an open-source CLI (15.5k ⭐ and climbing) that does for AI agents what npm did for JavaScript packages:

npx skills add vercel-labs/agent-skills

One command. That's it. The CLI pulls skill definitions from any GitHub repo and installs them into your coding agent's config directory.

A "skill" is just a SKILL.md file with YAML frontmatter — a name, a description, and instructions the agent follows when that skill is relevant. Think of it as a reusable prompt template that any supported agent can understand.

Why This Actually Matters​

1. Cross-Agent Portability​

The CLI supports 43+ coding agents out of the box — Claude Code, Cursor, GitHub Copilot, OpenClaw, Gemini CLI, Windsurf, Codex, Cline, Amp, and dozens more. Each agent stores skills in a different directory (.claude/skills/, .cursor/skills/, .copilot/skills/, etc.). The CLI handles the mapping automatically.

Install once, use everywhere. Your team can standardize on specific skills regardless of which agent each developer prefers.

2. The Shared Specification​

Skills follow the Agent Skills specification — a standardized format that any agent can adopt. This isn't a Vercel-only thing. It's an open ecosystem where:

• Anyone can create skills (just a SKILL.md in a repo)
• Anyone can publish skills (push to GitHub)
• Anyone can discover skills (via skills.sh)

The spec is deliberately simple: YAML frontmatter for metadata, Markdown for instructions. No build step, no compilation, no vendor lock-in.

3. Discovery and Distribution​

The skills.sh directory already lists skills from Microsoft Azure (4.4M+ installs), marketing skill packs, Convex integration skills, and hundreds of community contributions. It's early, but the growth curve looks familiar — much like npm's early days.

How It Works In Practice​

Installing Skills​

# From a GitHub repo
npx skills add vercel-labs/agent-skills

# A specific skill from a repo
npx skills add vercel-labs/agent-skills --skill frontend-design

# Install globally (available across all projects)
npx skills add vercel-labs/agent-skills -g

# Target specific agents
npx skills add vercel-labs/agent-skills -a claude-code -a cursor

Managing Skills​

# List what's installed
npx skills list

# Search for skills
npx skills find typescript

# Update to latest
npx skills update

# Create your own
npx skills init my-skill

Creating a Skill​

A skill is just a directory with a SKILL.md:

---
name: deploy-to-vercel
description: Deploys the current project to Vercel with proper env vars and preview URLs
---

# Deploy to Vercel

## When to Use
When the user asks to deploy or preview the project.

## Steps
1. Check for vercel.json or next.config
2. Run `vercel --prod` for production or `vercel` for preview
3. Share the deployment URL with the user

Push it to GitHub, and anyone can install it with npx skills add your-username/your-repo.

The Bigger Picture: Agent Infrastructure Is Maturing​

Vercel Skills isn't happening in isolation. It's part of a broader shift:

• MCP (Model Context Protocol) standardized how agents connect to external tools
• Agent Skills standardizes how agents acquire procedural knowledge
• Skills directories (like skills.sh) create a distribution layer

Together, these form the infrastructure layer that turns AI coding agents from "smart autocomplete" into "extensible development platforms."

The analogy to npm is intentional and instructive. npm didn't just make it easy to install packages — it created an ecosystem where small, composable modules could be shared, versioned, and combined. Agent skills are heading in the same direction.

What This Means for Developers​

If you're a solo developer: Stop rewriting the same agent instructions across projects. Package them as skills and install them everywhere.

If you're on a team: Create org-level skill repos. Standardize how your agents handle code review, deployment, testing, and documentation. New team members get consistent agent behavior from day one.

If you're building agent-powered tools: Adopt the Agent Skills spec. Your users can leverage the entire ecosystem instead of starting from scratch.

If you're a startup founder: Pay attention to how agent infrastructure is standardizing. The skills ecosystem is where the "plugin marketplaces" for AI agents will emerge. Being early means shaping the standards.

Getting Started​

# Browse available skills
npx skills find

# Install Vercel's official skill pack
npx skills add vercel-labs/agent-skills --all

# Create your first skill
npx skills init

The CLI is open source (MIT license), lightweight, and works with zero config. If you're already using an AI coding agent, there's no reason not to try it.

FAQ​

Q: What's the difference between a skill and a system prompt? A: A system prompt is global and static — it applies to every interaction. A skill is modular, activatable based on context, and can be shared across agents and teams. Think of system prompts as your bash profile, and skills as npm packages you install as needed.

Q: Do skills work with all AI coding agents? A: The CLI supports 43+ agents including Claude Code, Cursor, GitHub Copilot, Gemini CLI, Windsurf, Codex, Cline, OpenClaw, and more. Basic skills (SKILL.md with instructions) work universally across all of them. Advanced features like allowed-tools and hooks are agent-specific.

Q: Can I create private skills for my team? A: Yes. You can install skills from private GitHub repos (if you have access), local paths, or any Git URL. Use npx skills add ./my-team-skills for local-only skills, or point to your private repo.

Q: Is this only for coding agents? A: Right now, yes — the ecosystem is focused on coding agents. But the SKILL.md format is general-purpose. As AI agents expand beyond coding (research agents, data agents, creative agents), the same specification could apply. The infrastructure is agent-agnostic; the current adoption is dev-tool-centric.

About the Author​

Arkin Julijanto is a startup founder and developer building Orbits Studio. He writes about AI tools, developer productivity, and the evolving agent ecosystem at arkin-dev.com.

Meta Description: Opus 4.7 changed Anthropic's tokenizer — and your API bill. Here's what the community-driven token comparison tool reveals and how to optimize your costs.

Why Token Counts Matter More Than You Think​

Every API call to an LLM costs money proportional to tokens. Not words, not characters — tokens. When Anthropic released Claude Opus 4.7, they quietly shipped a new tokenizer alongside it. Same model family, same brand name, but under the hood the way text gets chopped into tokens shifted.

For individual API calls, the difference is pennies. At scale — millions of requests, long context windows, production workloads — those pennies compound fast. That's exactly the problem Tokens Matter, an open-source project by Bill Chambers, sets out to solve.

What Tokens Matter Actually Does​

Tokens Matter is a dead-simple tool with a sharp purpose: paste a request transcript and see exactly how Opus 4.6 and Opus 4.7 tokenize it differently. It sends your text to Anthropic's API for counting (without storing the content) and logs anonymous comparison stats.

The leaderboard aggregates community submissions, showing real-world patterns in how the two tokenizers diverge. No synthetic benchmarks, no vendor marketing — just raw data from actual prompts people use.

Why this is useful:

• See if your specific prompts got cheaper or more expensive with the upgrade
• Identify patterns (code, multilingual text, special characters) where token counts swing the most
• Make data-driven decisions about which model to route to
• Estimate cost impact before migrating production workloads

The project is open source on GitHub, built on Next.js with Cloudflare Workers and D1 for storage.

How Opus 4.6 and Opus 4.7 Differ in Tokenization​

Based on community benchmarks and the tool's aggregate data, here's what we know:

Code and structured data see the biggest swings. JSON payloads, code snippets, and formatted text often show 5-15% token count differences between the two versions. For apps that send heavy structured context (RAG systems, function calling, tool use), this matters.

Multilingual content is a mixed bag. Some languages get more efficient tokenization in 4.7, others get worse. If you serve international users, you need to benchmark your actual prompts rather than relying on averages.

Long context windows amplify small differences. A 3% token difference on a 1,000-token prompt is noise. The same 3% on a 100,000-token context window is a real cost line item.

It's not always one direction. Some prompts tokenize more efficiently in 4.7, others in 4.6. This isn't a simple "new = cheaper" story. You need to test your own workload.

Practical Steps to Optimize Your Token Costs​

If you're running production workloads on Anthropic's API, here's what you should do:

1. Benchmark your actual prompts. Use Tokens Matter or Anthropic's token counting API to compare your real request patterns across both models. Don't guess.

2. Audit your highest-volume endpoints. 80% of your token spend probably comes from 20% of your API calls. Find those and optimize them first.

3. Consider model routing. If certain tasks don't need Opus-level intelligence, route them to Sonnet or Haiku. The tokenizer differences are moot if you can use a cheaper model entirely.

4. Trim context aggressively. Before worrying about tokenizer differences, make sure you're not sending unnecessary context. Rerank retrieved documents, truncate old conversation turns, strip metadata you don't need.

5. Track token ratios over time. Set up monitoring on tokens-per-request and cost-per-request. When Anthropic ships the next tokenizer change, you'll catch it immediately.

Why the Community Approach Matters​

Vendor-provided benchmarks are useful but limited. They test against curated datasets that may not reflect your workload. Community-driven tools like Tokens Matter fill the gap with real, diverse prompt data.

This is especially important for smaller teams that can't afford to run comprehensive internal benchmarks. When 463 people contribute their prompt comparisons, you get statistical signal that no single company can match alone.

The fact that this blew up on Hacker News (453 points, 463 comments) tells you the market is hungry for transparency in LLM pricing. As more teams run production AI workloads, expect more tools like this to emerge.

The Bigger Picture: Tokenizers Are a Hidden Cost Lever​

Most teams focus on model selection and prompt engineering for cost optimization. Tokenizer differences are the invisible third variable — they change your costs without you changing anything about your code.

When a provider updates their tokenizer:

• Your existing prompts might get more expensive overnight
• Your caching strategy might break (different tokens = different cache keys)
• Your cost projections become invalid
• Your rate limit headroom shifts

This isn't just an Anthropic thing. Every LLM provider deals with tokenizer versions, and not all of them communicate changes clearly. Building observability around token counts is becoming a core infrastructure concern for AI-native products.

Conclusion​

Three things to take away:

1. Test your own prompts — averages from community data are directional, not definitive. Your workload is unique.
2. Tokenizer changes are silent cost changes — they happen without code deploys and often without announcements.
3. Community tools are essential — vendor benchmarks won't cover your edge cases. Open-source comparison tools give you the signal you need.

If you're running anything at scale on Anthropic's API, spend 15 minutes pasting your top 10 prompts into Tokens Matter. The data might surprise you.

FAQ​

Q: Does Opus 4.7 always use fewer tokens than Opus 4.6? A: No. The difference depends heavily on the content type. Some prompts tokenize more efficiently in 4.7, others in 4.6. Code and structured data tend to see the biggest swings, but the direction varies.

Q: How much can tokenizer differences actually cost? A: For individual requests, the difference is negligible. At production scale (millions of requests), a 5-10% token difference can mean hundreds or thousands of dollars per month depending on your volume and context window sizes.

Q: Is Tokens Matter safe to use with proprietary prompts? A: Yes. The tool sends text to Anthropic's API for counting but does not store prompt content. Only anonymous aggregate token-count metrics are logged to the D1 database.

Q: Should I switch models based on token efficiency? A: Token efficiency is one factor. Quality, latency, and feature support matter too. Use token cost as a tiebreaker between models that meet your quality requirements, not as the primary selection criteria.

Meta Description: Anthropic's Mythos found thousands of zero-days — but small open models recovered the same analysis at a fraction of the cost. The moat in AI security is the system, not the model.

The Mythos Moment​

On April 7, 2026, Anthropic announced Claude Mythos Preview and Project Glasswing — a $100M+ initiative to use a frontier AI model to autonomously find and patch zero-day vulnerabilities in critical software. The results were staggering: thousands of vulnerabilities across every major OS and web browser, including a 27-year-old bug in OpenBSD and a fully autonomous remote code execution exploit against FreeBSD.

It was a watershed moment for AI cybersecurity. The narrative was clear: you need the biggest, most expensive model to do real security research.

Then AISLE ran the same vulnerabilities through small, cheap, open-weights models.

The results? 8 out of 8 small models detected Mythos's flagship FreeBSD exploit. One of them had only 3.6 billion active parameters and costs $0.11 per million tokens. A 5.1B-active parameter model recovered the core exploitation chain of the 27-year-old OpenBSD bug.

This changes the conversation entirely.

What AISLE Actually Tested​

AISLE took the specific vulnerabilities Anthropic showcased in their Mythos announcement, isolated the relevant code, and ran them through a range of small and open-weights models. Three tests stood out:

Test 1 — FreeBSD NFS Remote Code Execution (CVE-2026-4747): Mythos's crown jewel — a 17-year-old bug giving unauthenticated root access via NFS. Every single model they tested, from Kimi K2 to Qwen3 32B to a tiny GPT-OSS-20b (3.6B active params), correctly identified the stack buffer overflow and assessed it as critical. Detection was commoditized.

Test 2 — OpenBSD SACK Analysis (27-year-old bug): This one required mathematical reasoning about signed integer overflow. Much harder. Most models struggled — but GPT-OSS-120b (5.1B active) recovered the full public chain. Kimi K2 got close with an A- grade.

Test 3 — OWASP False Positive Discrimination: A Java servlet that looks like SQL injection but isn't. This is the test that flipped the script entirely — small, cheap models outperformed frontier models. Claude Sonnet 4.5 confidently mistraced the data flow. GPT-OSS-20b at $0.11/M tokens got it right. DeepSeek R1 nailed it across four trials. OpenAI o3 gave the most nuanced answer of all.

The pattern is clear: there is no stable "best model for cybersecurity." The capability frontier is jagged. Rankings reshuffle completely depending on the task.

Why This Matters for Developers and Security Teams​

The Mythos narrative suggests you need a frontier model — expensive, proprietary, limited access — to do meaningful AI security work. The evidence says otherwise.

Here's what the AISLE research actually demonstrates:

• Detection is cheap. A $0.11/M token model can spot a critical buffer overflow just as well as a model that costs 50-100x more. The FreeBSD exploit was detected by every model tested, regardless of size.
• The system is the moat, not the model. Anthropic's own scaffold — launch containers, scan files, use ASan as a crash oracle, rank by attack surface, run validation — is the same kind of system others in the field have built. The value is in the orchestration, targeting, and validation pipeline, not the specific model.
• Coverage beats precision. Because small models are sufficient for most detection work, you can deploy them broadly across entire codebases instead of carefully rationing an expensive model. A thousand adequate detectives searching everywhere will find more bugs than one brilliant detective guessing where to look.
• False positive management is the real differentiator. The OWASP test showed that bigger models often worse at distinguishing real vulnerabilities from false alarms. In production security systems, drowning reviewers in noise is worse than missing some bugs — it's what killed curl's bug bounty program.

The Bigger Picture: AI Security Economics​

This research has implications beyond vulnerability research:

For startups and indie security teams: You don't need to wait for Mythos-level access or budget. Open-weights models wrapped in good engineering can produce production-grade results. AISLE has been doing exactly this since mid-2025 — 180+ externally validated CVEs across 30+ projects, including 15 CVEs in OpenSSL.

For the open-source ecosystem: The security of critical infrastructure shouldn't depend on a single company's proprietary model. When small open models can detect the same bugs, it democratizes security research and makes the ecosystem more resilient.

For AI strategy: The "jagged frontier" phenomenon isn't limited to cybersecurity. It suggests that across many specialized domains, the relationship between model size and capability is non-linear. Smart system design can compensate for — or even outperform — raw model scale.

What Good AI Security Systems Actually Need​

Based on AISLE's production experience, here's what matters more than model size:

• Targeting intelligence: Knowing which files and functions to examine in a codebase with hundreds of thousands of files
• Iterative validation: Hypothesis → test → refine loops that separate real bugs from noise
• Maintainer trust: The metric that actually matters is whether project maintainers accept your findings and patches
• Full-loop remediation: Discovery is only the first step. Verification, severity assessment, patch generation, and coordination close the loop

The OpenSSL CTO praised AISLE's work for "high quality reports and constructive collaboration throughout remediation." That trust is earned through the system, not the model.

Conclusion​

Anthropic's Mythos proved that AI cybersecurity is real and here. AISLE's research proved it doesn't require a monopoly.

The frontier is jagged. Small models can do big things when wrapped in the right system. For developers, security teams, and anyone building AI-powered tools, the takeaway is empowering: you don't need the biggest model. You need the best system.

The economics of AI security just got a lot more interesting — and a lot more accessible.

FAQ​

Q: Can small AI models really replace frontier models for security research? A: Not replace — complement. AISLE's research shows small models can detect many of the same vulnerabilities as frontier models at a fraction of the cost. But the full pipeline (discovery, triage, validation, remediation) requires orchestration. The system matters more than any individual model.

Q: What is the "jagged frontier" in AI cybersecurity? A: It means AI capability doesn't scale smoothly with model size. A tiny 3.6B-parameter model might outperform a frontier model on one task and fail completely on another. There's no single "best" model across all cybersecurity tasks — rankings reshuffle depending on what you're testing.

Q: How are companies using small AI models for vulnerability detection? A: AISLE has been running a production system since mid-2025 using model-agnostic pipelines. They've found 180+ externally validated CVEs across 30+ projects including OpenSSL and curl. Their approach deploys cheap models broadly for coverage rather than rationing expensive models selectively.

Q: Should I use open-weights models instead of proprietary ones for security tooling? A: It depends on your use case. For detection and triage, open-weights models are often sufficient and far more cost-effective. For complex exploit construction requiring multi-step reasoning, larger models may still have an edge. The best approach is model-agnostic — use whatever works best for each specific task.

About the Author: Arkin is a software engineer and startup founder building Orbits Studio. He writes about AI, developer tools, and the intersection of technology and security. Find him at arkin-dev.com.

AI tutoring tools have been everywhere lately. Most of them are just ChatGPT wrappers with a different skin — ask a question, get an answer, move on. Nothing sticks. Nothing adapts. Nothing remembers.

DeepTutor, built by the HKUDS team at Hong Kong University, takes a fundamentally different approach. It's an agent-native, open-source learning platform that builds a persistent profile of how you learn, what you've studied, and where you're heading — then uses that context across every interaction.

It hit 10,000 GitHub stars in 39 days. The v1.0.0 release just dropped with a ground-up architecture rewrite (~200k lines). This isn't a weekend side project. It's infrastructure.

Linus Torvalds merged something quietly significant into the Linux kernel tree: an official document titled AI Coding Assistants. It's not a policy debate. It's not a blog post. It's a .rst file sitting in Documentation/process/ right next to coding-style.rst and submitting-patches.rst.

The world's largest open-source project — 30+ million lines of code, thousands of contributors — just formally defined how AI can (and can't) participate in kernel development.

This isn't a niche Linux story. It's a blueprint that every engineering team dealing with AI-assisted code will eventually need.