What an AI Feedback Loop Looks Like

this post is part of a series about creating production-grade maintainable AI-first projects, using AI-first design patterns and frameworks. In the previous post I mentioned that an internal AI feedback loop will be central to all our AI-first design patterns. But “AI feedback loop” might mean different things to different people - so this lightweight post focuses on giving an example to make it concrete. We will implement a small (but realistic) project. The project is set up so the agent has an internal feedback loop - it has instructions that tell it to use a loop, and it has a clear way to create effective tests and run validations (the tests it creates, type-checking, linter). We’ll see how it makes mistakes, finds them and self-heals. ...

October 4, 2025 · Shai Geva

10 Ways To Shoot Yourself In The Foot With Tests

This is a series of posts, following a talk I gave (twice - at Pycon-US 2023 and Pycon-IL 2024), about testing best (and not-so-best) practices. The talk shares 10 practices that I had bad experience with, along with ways of avoiding them. The main objective of the post series is to help you write tests that have a better ROI. I’ll discuss different practices, different ways that we can work. These practices affect us by changing the properties of our tests: ...

May 16, 2025 · Shai Geva

Footgun #2 - Untested Tests

This mini-post is part of a series about good testing practices, which I also presented at a couple of conferences. Here it is in PyCon US 2023 Sometimes our tests lie to us. We have a test that was supposed to protect us from some bug, but that bug happened after all. Of course, what happened was that we made a mistake, and the test didn’t really verify what we thought it does. ...

October 5, 2024 · Shai Geva