Member-only story
April 2025 — Agentic AI Review
Everyone has been talking about AI development and “Vibe Coding,” so I decided to take a week or so to dig into the technologies out there and familiarize myself with the latest options, capabilities, and limitations. Like many engineers (and PMs), I hoped to find some way to insert AI into my development process as a force multiplier. This area is changing incredibly rapidly — two of the tools I reviewed weren’t available last month, and Codex CLI was made publicly available just last week.
To evaluate their autonomy, I created the AI Coding Agent Autonomy Scale.
This is by no means a complete or unbiased appraisal — my bias and evaluation are shaped by my experience as an independent software developer working solo. I’ve already replaced a lot of research and documentation work with ChatGPT and Copilot (which is great at keeping README and CONTRIBUTING documents up to date). I was hoping to increase my coding output with more agentic, higher-level AIs. Here’s what I tried:
Copilot Agent Mode
Copilot has been around for a while as “Ask,” a chat box where you can query your code (with a few backend models to choose from). They added “Edit,” which allowed it to modify files. Recently, they announced and made available “Agent” mode, which can edit files and run some terminal commands (many require manual…
