Alex
FLOSS virtualization hacker, occasional brewer
member since 9 Jun 2023 09:57
I wrote a blog post for the project this week detailing some of the challenges the QEMU project, in common with other FLOSS projects, are seeing with AI generated bug reports. The dilemma being that the LLM’s have got to the point they generate better bug reports than a lot of humans do.
If the bugs where being found in code that is used heavily in production I’d agree. Spamming the tracker with 100’s of bugs in random unmaintained areas of the code base just wastes everyone’s time. Perhaps as a project we should be more aggressive in deprecating code but the threat profile for someone running firmware blobs from their camera is very different from people launching untrusted guests on their shared hosting infrastructure.
I do run a bot that can do the initial triage, it doesn’t solve the problem of bugs stacking up in unmaintained areas of the code. I have experimented with auto-fixing the bugs with an LLM but currently the project policy is no AI contributions at all. We are in the process of debating a new policy which might allow for simple fixes to be done with AI.
Gymnastics might not be a bad alagory for catalysis. It’s all about getting the molecules lined up to make the reactions easier than relying on heat and random collisions to get you there.
I’m sure Reagan deserves a lot of the criticism and he was possibly in early stage Alzheimer’s in his second term. However I mostly remember him being the president in charge when the cold war ended and Eastern Europe was finally liberated.
In the discussions our project are having about how to approach LLM use in the community there has been a lot of consensus in the human taking ownership. Ultimately it’s the human who has to direct the tool and if they have little interest other than blindly hitting accept then we’re not interested in the contribution.
Some of the AI generated bug reports we get are good because the author had verified them, added a test case and at least reviewed if it is guest trigger able. Yesterday’s flood however was low effort spam.
You don’t think the creators should get paid? Do you know how much effort goes into making quality videos?
Ironically it’s probably because so many of the slop videos try and copy their style. However I would have thought YouTube would be more proactive with a channel with 25M subs.
Out of an abundance of caution we don’t let the agent close bugs or assign CVEs but tell it to defer to a real human for that. I’ll have to experiment with how we can do a better job of detecting duplicates because I’m sure a fair number of the issues are.