Until now this repository has collected a little over three thousand commits, and agents wrote most of them. Whenever something went wrong, I blamed the agent, usually out loud and usually late in the evening (and usually with Russian profanity).
Last week I scanned the changelog from the beginning. The agent came out of it rather better than I did. In almost every case it had done exactly what I asked. The trouble was what I had asked.
Two old questions fit moments like this, and each is the title of a long novel: who is to blame, and what is to be done? Neither novel was written with pull requests in mind.
Who is to blame
I once wrote a prompt for our pricing pages that told the model to be specific with numbers. A few lines further down, the same prompt listed phrases that would cost a page quality points, and "contact for pricing" was one of them.
For vendors who publish no prices at all, I had given two orders that could not both be followed. A person would have come back and asked which one I meant. The agent found the only way to obey both: it invented the numbers.
Fifty-seven published pages told readers what buyers had actually paid, "according to Vendr transaction data from 45 deals". We hold no such data, and none of our sources mentions Vendr.
The worst part: these pages scored 100 out of 100. Invented numbers are specific and confident, so they passed every other check we had.
Of course, this is fixed now, but a bad taste lingers. The key point of this example is that it was I who wrote the prompt that could not be obeyed...