We've had a couple of incidents (all today) of clbot doing things that we don't expect it to do, specifically:
It repeatedly submitted a partial chain from the "bottom" before all CLs were ready. This is actually desirable, but surprising.
It rebased the "top" of a chain out of the chain (!) and attempted to submit it, but failed because CI failed (there was a reason the CL was part of a chain after all).
In the logs of the service it seems like its series (i.e. CL chain) detection is failing for some (as of yet) mysterious reason and returning smaller chunks than expected. It also seems to be ignoring Gerrit "submittability".
What's weird about this issue is that literally nothing was changed in the service code, not even dependency versions (which are pinned of course). We deployed whitby a few days ago to a new channel, and I suspect that whatever happened was introduced through the channel bump changing some behaviour in either Gerrit or gerrit-queue (JVM, Go compiler, idk ...).
For some reason this only happened on the day that this issue was reported, which leaves a host of weirdly fun possibilities for what caused it (calendar dependencies? where? something extraordinary happening on whitby that day?)
tazjin at 2022-06-11T16·26+00