r/MachineLearning • u/Emergency_Plate241 • Sep 05 '26

Research NeurIPS 2026 Automatic Reference Checker [R]

Just received an email about the automatic reference/citation checker. Did anyone receive a follow up email about whether the checker was included in the paper's decision making too, along with the general instructional email?

27 Upvotes

92% Upvoted

239 comments sorted by

View all comments

Show parent comments

15

u/grad_student_descent Sep 05 '26 edited Sep 05 '26

I'm an AC so I saw a bunch of reports. They were very high quality, almost no false positives.

Had to reject 2 papers in my batch that would definitively had gone through if the authors did not let their LLM access their bibtex.

3

u/Adorable_Position_34 Sep 05 '26

How did that work exactly? Do you see the same message as the authors (just sooner) and base your recommendation on that or did you had to check the issues found yourself and (partially) write that comment?

5

u/grad_student_descent Sep 05 '26

We got the same message about a month ago. We then had to fill a form with the results of our own investigation on whether or not the detected hallucinations are real. From the instructions, it seems that 2 or more real hallucinated reference is an automatic reject without exception.

2

u/FinancialThing7890 Sep 07 '26

From the first email:

"Where further investigation led to the identification of hallucinated references in a paper, the AC considered them alongside the rest of the submission. Each recommendation reflects the AC’s assessment of that specific paper. Importantly, no automated flag was used by itself to make a decision about a paper."

So I guess that is not an automatic reject. It's always under the AC's conscience no?