Posts

(first post in a series) Like many others, I spent a lot of time thinking about AI and software development. I belong to the camp that believes that AI is a total paradigm shift - it’ll redefine the ecosystem and what it means to create software, and it’ll be the deepest change we have seen to date. My own “flavor” of thinking about this is to try, from an engineering / implementation perspective, to understand what that change could look like. This blog series will dig into this and share my thoughts and conclusions from experimentation. I’ll talk about both: ...

This is a series of posts, following a talk I gave (twice - at Pycon-US 2023 and Pycon-IL 2024), about testing best (and not-so-best) practices. The talk shares 10 practices that I had bad experience with, along with ways of avoiding them. Starting with simple (but useful!), and moving on to more complex ideas: There are no tests (warm up) Untested tests The tests are not isolated No locality of behavior Unclear language Testing too many things Improper test scope Test doubles everywhere Slow tests Wrong priorities Videos and slide decks from the talks ...

(this mini-post is part of a series about good testing practices) Just like with product code, if we put too many things in the same place we get a mess. My rule of thumb is to try hard to test a single fact about the behavior of the code. And it helps if I use these specific words mentally. ...

(this mini-post is part of a series about good testing practices) Another problem that makes it more difficult to understand tests is unclear language. Two guidelines that help me deal with this: We want to use decisive language We want the language to be specific and explicit Suppose we have a book store and we’re testing the functionality for editing a book. Let’s see some examples of test phrasing: ...

(this mini-post is part of a series about good testing practices) One testing problem that doesn’t get enough attention in my opinion is tests that don’t have locality of behavior. By that I mean cases where a test is broken down into different parts in a way that makes understanding more difficult. This is important in every type of code, and tests are no exception. Example: non-local data Consider this test: ...

(this post is part of a series about good testing practices) Writing tests that are not isolated is a sure way to create unnecessary work for ourselves. By “tests that are not isolated”, I mean tests that sometimes have a different outcome (failing / passing) if we run only a subset of them, if we run them in a different order or if we run them in parallel. Why is this a problem? Let’s say we have 30 tests, and test 24 passes if we run it individually but fails if we run the entire test suite. ...

(this post is part of a series about good testing practices) Sometimes our tests lie to us. We have a test that was supposed to protect us from some bug, but that bug happened after all. Of course, what happened was that we made a mistake, and the test didn’t really verify what we thought it does. As it turns out - when we write a test, it’s a good idea to spend a little effort to verify the test actually works. To make sure that if the bug happens, the test does indeed fail. ...

(this post is part of a series about good testing practices) This is a “warm-up footgun” to the blog post series. The easiest way to shoot yourself in the foot, testing-wise, is to have no tests at all. In my experience, writing any tests often helps us - even if these tests are not well-written, and even if they’re just a drop in the sea. There are a few reasons I noticed, why moving from no tests at all to even one test for some area of the code is useful. ...