I Think You Might Be Fooling Yourself with AI

If you are enthusiastic about AI (LLMs), you may be fooling yourself. Do not consider this an insult, but a kind-hearted warning.

Remember what Richard Feynman said:

The first principle is that you must not fool yourself and you are the easiest person to fool.

source (backup)

He made this statement as part of the Caltech's 1974 commencement address. In this speech he strongly advocated for proper scientific integrity.

I feel it can be difficult to maintain proper 'scientific integrity' on a personal level as AI is a fast-moving target. The tools feel unreasonably capable, using an LLM makes you feel more productive when writing code for instance.

Yet as of today, there is no independent scientific evidence (that I'm aware of) that people or organizations are indeed more productive when using AI.

Are you really more 'productive'?

Meanwhile, a study (late 2025) seems to report that although participating developers felt they completed tasks faster using AI, they where around 19% slower.

Using AI may feel faster and more productive, but you might be fooling yourself. You can never go back in time and measure yourself doing the task without using AI tools and compare the results.

... continue reading