『Agents and Engineers』のカバーアート

Agents and Engineers

Agents and Engineers

著者： Dan Gerlanc
無料で聴く

The podcast about Agentic AI and Software Engineering. Each episode is a conversation with people whose daily lives most intersect with AI and agentic systems. Join me as I follow the stories, the behind-the-scenes, and the real people behind the code.© 2026 Dan Gerlanc LLC 経済学
エピソード
  • Skills, Context, and Trust: The New Agentic Coding Stack

    Skills, Context, and Trust: The New Agentic Coding Stack
    2026/07/07
    Dan and Jonathan Bown open with the talk Jonathan gave at ODSC, "Practical Agent Ops: From POC to Prod with MLflow 3.0." MLflow 3.0 arrived last summer as the first stable release built for generative AI rather than traditional machine learning, and Jonathan's team used it to build an agent for pre-enrollment students. The centerpiece of that work was evaluation-driven development. Instead of jumping straight into a working prototype they aligning the business up front on what quality actually looks like before signing off on a model with inherently non-deterministic output.The turning point came from an overlooked asset: a data science team had already assembled 150 ground truth examples in an Excel file, untested and set aside while engineers focused on code. Jonathan's team paused the coding work and ran a simple foundation model against those examples first, landing at what amounted to a coin flip of useful versus hallucinated answers. From there they refined the examples with the business, loaded them into MLflow's evaluation datasets built from live traces, and iterated by versioning prompts and agent configurations.Tooling came up repeatedly. MLflow's open source repo now ships a skill file that plugs into coding tools like Claude Code, which Jonathan called a game changer for keeping up with an API that changes at roughly a release a month. The Databricks AI Dev Kit, released around March, bundles skills for the Databricks SDK, CLI, data engineering, and analytics work, usable either inside Databricks' Genie Code pane or in outside tools such as Claude Code, AWS Kiro, or Google Antigravity. Jonathan said installing it produced a dramatic jump in output accuracy compared to coding assistants working from stale or incomplete context about Databricks and MLflow APIs.Dan raised the idea that LLMs and agentic tools are becoming users of software in their own right, alongside humans, and Jonathan tied that to broader changes at WGU: more of the business, not just engineers, now writes system prompts and builds their own copilot-style agents. His own day to day has moved from core development toward AI enablement, meaning security review, best practices, and helping non-technical staff adopt evaluation-driven habits for the prompts and agents they build themselves.Jonathan's path to WGU ran through Pentara, a biostatistics consultancy, and Zions Bancorporation, where he did quant finance work before a stint simulating financial products for WGU students. He became a founding member of WGU's MLOps team in 2023, when the university's machine learning was still traditional work like random forests and ensembles for predicting student outcomes, well before Databricks had built out MLOps tooling. Dan connected this to Hamel Husain's essay The Revenge of the Data Scientist, and Jonathan agreed that evaluation-driven development brings the work full circle: checking evals, correctness, tool call accuracy, and safety is the generative AI analogue of checking a confusion matrix.The pre-enrollment agent's rollout became the clearest illustration of the method. The first release, a bare foundation model with no WGU context, drew heavy negative feedback from the employees testing it, some of whom wanted to shut the project down. Jonathan's team treated that feedback as fuel, folding the failed questions into an evaluation dataset and iterating until they reached roughly 82 percent correctness and near-total relevance, at which point the same employees became enthusiastic supporters. He credited MLflow's architecture for building subject matter experts directly into the agent ops workflow rather than treating evaluation as a purely technical exercise.Jonathan was candid about where his trust runs out. He does not trust a tool's first output even after a full planning session, citing a Kiro planning cycle from the day before that failed on the first try despite extensive back and forth. He is cautious about MLflow's fast release cadence outpacing its own skill files, and notably guarded about tools like OpenClaw and Claude Cowork that can reach into email or personal documents. Given how much effort WGU puts into protecting student data, he extends the same caution to his own personal information and limits what such agents can access.On his team, Jonathan resists banning AI-generated code or stigmatizing it in review, and instead pushes everyone toward reviewing code outside their usual specialty, using AI review tools like Amazon Q or GitHub Copilot as a starting point rather than a final answer. He pushed back on the idea that tool usage equals productivity, warning about AI slop and noting that some of the heaviest users he knows are not the most productive. The thread ties back to evaluation-driven development's real thesis: start from value, not from the tool, a point he illustrated with WGU's Academic Virtual Assistant pilot, where a surprising result showed that students chatting with the assistant were ...
    続きを読む 一部表示
    1 時間 17 分
  • When Software Gets Cheap, Focus Gets Expensive

    When Software Gets Cheap, Focus Gets Expensive
    2026/07/02
    Dan and Greg open with how agentic development has changed since the early days of Copilot. At the time, Greg was at GitHub, and he saw AI mostly help with boilerplate and editor completions. Cursor-style agents were the next widely-used advancement bringing session history and integrated team-wide practices. By June 2026, capable models and harnesses are common inside engineering teams, so the gap between teams increasingly comes from context engineering, repository structure, and whether old team shapes still align with the new ways of building software. For small teams and startups, the leverage of AI is a double-edged sword. Greg describes how SpecStory's original extensions required real sweat equity to reverse engineer chat-log formats across Cursor, Copilot, Claude Code, Amp, and other tools. Now, much of that surface can now be maintained by a fraction of one person's time. The danger is that easy MVPs can trick founders into believing they have validated a market. When the marginal cost of software falls, founders have to spend more of their scarce attention on demand, willingness to pay, distribution, and the routes to customers. The conversation turns to Greg's book, 25 Patterns in Agentic Engineering. He explains how he mined roughly 1,300 preserved SpecStory sessions and nearly 5,000 commits to extract durable patterns from his own agentic practice. Two patterns stand out. First, when code becomes free, verification becomes the bottleneck. Second, between agents turns, docs are the persistent API of the system. For Greg, as-built architecture documents are practical maps that let both humans and agents recover the shape of a subsystem without re-reading the entire codebase every time. Greg's development practice has changed accordingly. He favors trunk-based development and says his team uses almost no pull requests for everyday development, partly because agent-generated diffs arrive at a volume he does not want to review line by line. He prefers local agents over cloud agents that containerize the repo and open PRs later, because steering an agent while it runs keeps his mental model intact. Long unattended runs still make sense to him, but only when they start from a clear goal and a more detailed rider, with phased commits and verification points he can inspect after a walk or a night away. Dan and Greg also dig into coordination at larger scale. Greg is skeptical that issue trackers were ever clean or current enough to describe day-to-day engineering, but he sees issues becoming useful as specs with provenance and evidence that can be handed to agents. Personally, he runs several projects at once, usually three to five, with local agents in permissive modes, and rotates attention while long runs execute. That power is not free. He describes the dopamine loop of watching ideas come to life, the temptation to keep agents busy overnight, and the scarcity mindset created by subsidized access to frontier models. The episode closes with where Greg still does not trust the tools. Copywriting and visual design still require heavy human intervention because the models can blur rather than sharpen the message. He frames taste less as a mystical trait and more a selection amongst trade-offs and the ability to connect ideas in understandable ways. Coding has benefited from benchmarks and verifiable answers; much of the rest of the world is less tractable because there is no single ground truth for what "good" means. Full episode notes Click here to view the episode transcript. Chapters (00:00) - Introduction and guest background(00:55) - What agentic teams are running into(06:56) - Startup leverage, MVP traps, and maintaining SpecStory(09:26) - When software gets cheaper, distribution matters more(12:31) - Hand-written code, craft, and code as liability(16:21) - Mining 1,300 sessions into 25 patterns(19:01) - Verification and as-built architecture docs(23:55) - Co-writing docs with LLMs(25:15) - Keeping docs fresh through skills, Git, and verbose commits(27:50) - Trunk-based development for agentic teams(30:26) - Local steering versus cloud-agent pull requests(32:14) - Goal and rider plans, long runs, and Gas Town(35:52) - Replacing issue trackers with weekly docs(38:19) - Larger teams and issues as agent-ready specs(42:45) - Parallel projects and concentration limits(44:47) - Local agents, permissions, and risk judgment(46:57) - The cognitive pull of managing agents(51:58) - Scarcity, token costs, and model choice(58:57) - Copy, design, naming, and taste(01:05:04) - Why creative output resists verification(01:07:12) - Closing ⠀ Links from the show -------------------- Hardcore Agentic Engineering for builders who shipSpecStoryStoa25 Patterns in Agentic EngineeringAI Essentials for Tech ExecutivesMeditations on TechBeyond Code-CentricGoal EngineeringWebRTCCRDTTrunk-based developmentSteve Yegge's Gas TownDead ReckonDevinDORABearDeepSeekQwenYann LeCun ⠀ Guests ------- Greg Ceccarelli, ...
    続きを読む 一部表示
    1 時間 8 分
  • From Supervising AI to Building Systems for It

    From Supervising AI to Building Systems for It
    2026/06/18
    Dan and Eleanor open by discussing how fast software engineering has changed. In the last six months, Eleanor's practice flipped from treating AI as a messy assistant that needs close supervision to building systems that put the agents on the path to success. She now writes essentially no code herself, arguing that the models have become good enough that her involvement mostly makes the results worse. This journey starts from babysitting agents locally to delegating to async, cloud-based agents like GitHub Copilot, Cursor, Devin, OpenHands, or Factory. Eleanor warns that the home-grown terminal "loops" everyone is building right now are great for learning but too brittle to scale. Next up, what does an agent engineering system actually need? Eleanor recommends starting with a sandboxed, execution environment (usually containers), careful configuration over how the agent reaches the outside world (MCP servers and selective network access), a way to see across multiple repositories, and layered rules via AGENTS.md and skills. Eleanor makes the case that async delegation is a forcing function for better specifications. Deterministic feedback like static analysis and test suites are the single biggest factor in work quality because "you can't control AI with AI." She has moved to fully test-driven development and notes that current-generation models no longer find unintended workarounds to tests (e.g., deleting them) the way Claude 4 and early GPT-5 once did. Dan and Eleanor turn to adoption and skills, including how to get better at using AI with deliberate practice. Eleanor explains why she moved using Python, which she was most familiar with from use over her career, to statically typed languages like TypeScript and Go for agent work, why supply chain risk at her healthcare company has her questioning every dependency, and why she dislikes the term "junior developer." Curiosity and systems thinking, not tenure, are what matter now. The episode closes on verification and scale. Eleanor distrusts any output she can't verify, doesn't miss hand-writing code, and argues that inventing new ways to verify, including more formal methods, is the real bottleneck now that models are cheap and strong. On team size, she pushes back on the "small teams" consensus, pointing to the success of large open-source communities. Eleanor remarks that software development has become a sub-branch of systems engineering, and anyone not practicing this now will be shocked in a matter of months. Full episode notes Click here to view the episode transcript. Chapters (00:00) - Introduction(00:58) - The flip: from supervising AI to getting out of its way(03:14) - Cloud-based agents vs. rolling your own(06:08) - The primitives every agent system needs(07:43) - Why async delegation beats local babysitting(11:02) - Writing specs: Codex, Repo Prompt, and markdown(12:21) - Guardrails: AGENTS.md, skills, and deterministic checks(14:19) - Going fully test-driven(17:19) - How engineers really adopt (and hide) AI(19:38) - Getting better through deliberate practice(21:12) - From experiment to reusable skill to library(24:26) - Choosing a language: Python, TypeScript, Go(26:56) - Supply chain risk and distributing specs(29:06) - Beyond 'junior': curiosity over tenure(31:03) - Systems thinking as the durable skill(38:02) - Where Eleanor still doesn't trust AI(39:40) - Not missing the keyboard(42:43) - Keeping up with a fast-moving field(44:50) - What teaching reveals(48:27) - Verification as the real bottleneck(50:41) - Team size and open source at scale(55:48) - Closing: take agents seriously Links from the show -------------------- GitHub Copilot coding agentDevinOpenHandsFactoryCodexRepo PromptAGENTS.mdModel Context Protocol (MCP)Anthropic 'when AI builds itself'LovableVercelFormal verificationUMLJimini Health Guests ------- Eleanor Berger, Member of the Technical Staff, Jimini Health WebsiteLinkedInX Follow the podcast ------------------- LinkedInThreadsInstagramTikTok Follow Dan Gerlanc ------------------- XLinkedInThreadsBluesky
    続きを読む 一部表示
    1 時間
adbl_web_anon_alc_button_suppression_t1
まだレビューはありません