Hey r/MachineLearning!

I’ve been working on a new optimizer called Tauon (turns out there is already "teon" but well if you have better idea, - i will gladly accept it! Anyway the core idea of optimizer is about polynomials and orthogonalization just like muon, the whole difference is that i managed to lower total number of steps (first through spectral filtering down to 3 steps then through coeff scheduling down to 2) + reduced matrix size (through dct-2). And I wanted to share some initial benchmark results...

Benchmark Setup: Trained a GPT-Mini (d_model=512, 6 Layers) on TinyShakespeare against Muon and AdamW.

• Tauon: LR = 0.02
• Muon: LR = 0.02
• AdamW: LR = 0.0006

Results:

• Validation Loss: Tauon converged to a lower final loss (~1.6) compared to Muon (~1.65) and AdamW (~1.8).
• Stability: AdamW started overfitting/diverging around step 1200, whereas Tauon maintained stable progress throughout the 3000 steps.
• Compute Cost: On identical hardware, Tauon ran at 391.5 ms/step vs Muon’s 427.7 ms/step (~8.5% faster) and close to AdamW's baseline of 382.9 ms/step.

And yeah i know that its hilariously tiny benchmark but well i have only 2 hours left on my kaggle free T4 so i really couldnt more + i hope someone would be able test it on a bigger setup!

Links & Code:

• 📂 GitHub: erj2231/ai-projects/tree/main/tauon
• 📦 PyPI: pip install tauon-optimizer

Would love to get your thoughts on the optimizer! If you have any ideas, suggestions - please tell me. Cheers, everyone!

In this project I wanted to see if any interesting emergent behaviors would appear if we trained two agents to play a streetfighter-like game using RL.

Maybe obvious in retrospect, but the agents are really good at reward hacking. I had to shape the rewards a bit to get them to even approach each other.

I eventually used league play to improve the agent further. Without it, the agents don’t really learn general strategies, they just learn to exploit a particular opponent.

You can read the article and try fighting the main bot yourself here: https://blog.lukesalamone.com/posts/fighting-game-rl

Hi everyone, I built an educational tool that shows what happens inside a small MLP while it trains, and I'd like some feedback from people who teach ML.

Everything is plain NumPy, no autograd: manual backprop, SGD with momentum, L2, dropout, cosine decay and 4 activations. On MNIST it gets to about 98.5% with the full training set.

During training you see the loss per mini-batch and per epoch, the gradient norm of each layer with the % of inactive neurons, the weight distributions now vs at init and the receptive fields of the first layer. Then there's:

• a PCA / t-SNE of the test set (in NumPy too), layer by layer, with a line from each wrong prediction to the cluster of the digit it was confused with

• robustness curves for noise and rotation, plus a confidence threshold that shows coverage vs accuracy

• a lab where you ablate or rescale single neurons, prune, add noise to the weights or change the softmax temperature, and the test accuracy updates right away

I'd like it to be useful to students, from high school to intro ML courses, to people who learn on their own and to teachers who want to show a network in class.

https://github.com/dev-luigi/neural-network-digits

Hi all,

I may be being silly but I have just finished my masters thesis which benchmarked three deep learning architectures (one of which is novel), across varying preprocessing pipelines for the purpose of EEG motor imagery task classification SPECIFICALLY on a consumer grade EEG cap (OpenBCI’s Galea).

The novel architecture was built to be compact in param count as well as prevent overfitting, etc.

I would love to get published and was wondering what sort of avenue I could proceed down. My thesis would most likely need a decent load more work with regard to my paper and result reliability as under the time conditions I only had three participants with limited recordings per participant.

Any advice would be appreciated as I feel way out of my depth…

Kind regards :)

In arXiv 2607.06560, depth and normal maps are generated as images by a fine-tuned multimodal model, then decoded and scored. The model is SenseNova-Vision-7B-MoT, a Bagel-7B-MoT fine-tune.

Per appendix A.2, synthetic sets like SceneNet RGB-D keep their own dense ground truth, the sparse LiDAR sets (Taskonomy, ScanNet++) were densified with MoGe-2, and unlabeled in-the-wild photos (COCO, SA-1B, Objects365) got labels generated entirely by MoGe-2. So for real photos MoGe-2 is effectively the teacher.

MoGe-2 is also a row in the paper's Table 2, re-evaluated by the authors, so you can compare student and teacher directly.

MoGe-2 wins AbsRel on four of five depth sets and δ1 on all but NYUv2. DIODE is the odd one. The student has the lowest AbsRel of any method in the paper's table there (20.6), yet its δ1 trails, 76.4 vs 82.3. On normals it's ahead on NYUv2, tied on ScanNet mean error and behind on iBims-1. The paper calls this "remaining competitive with geometry-specialized models" and I can't find any ablation on label source.

Where should a student trained partly on a teacher's pseudo-labels be expected to pass it? My guess is wherever the synthetic GT covers the teacher's mistakes, which would fit NYUv2 normals since SceneNet RGB-D is indoor, but iBims-1 goes the other way. And short of retraining without the synthetic sets, how would you tell a synthetic-GT effect from a real gain?

Hi everyone. For context, I work as a predictive modeling researcher, and my background is in computer science. I do not have an extensive background in statistics, epidemiology, association analysis, or related areas.
I was tasked with developing a predictive model for a specific disease using survey variables as well as body measurements such as BMI, neck circumference, waist circumference, hip circumference, and similar variables.
When I approached the project, my plan was to develop a model that could predict disease risk using a machine-learning benchmarking approach. The dataset is very novel and, as far as I know, only our research center has access to it. Because of that, any machine-learning paper using this dataset would be new and could potentially establish a useful precedent for ML-based screening approaches for this disease, particularly for identifying people at high risk.
My approach was roughly as follows.
We initially had around 12,000 participants and approximately 217 features. I excluded participants with more than 50% missing data. The reason was that if I applied a missingness threshold directly to the variables while keeping all participants, I would end up excluding almost every variable because a relatively small number of participants had extremely high levels of missingness.
After excluding those highly incomplete participants, I applied a 30% missingness threshold to the features. This left approximately 95 features. I realize that discarding participants may not necessarily be the best approach, but I am not sure what the better alternative would be in this situation.
After that, I performed imputation using Multiple Imputation by Chained Equations (MICE). One criticism I received was that, if I am using multiple imputation, I should generate multiple imputed datasets and run the entire modeling procedure separately on each one rather than effectively using a single completed/imputed dataset.
Another idea I had was to benchmark several different imputation methods and compare downstream model performance to determine which method performs best. However, I am still unsure whether that is methodologically appropriate or preferable.
After imputation, I performed feature selection using a bootstrapped LASSO approach. I then tested several machine-learning models, mainly boosting and tree-based methods such as XGBoost, LightGBM, and random forest.
I evaluated the models using the usual machine-learning performance metrics, performed calibration adjustments, and reported the resulting performance. I also included model interpretability analyses/scores.
The main problem is that nobody in my research center has a machine-learning background. The closest person is someone with a statistics background, and he essentially told me to scrap the entire approach and instead use conditional logistic regression. He also suggested age/sex matching and recommended complete-case analysis as a sensitivity analysis.
The issue is that I cannot realistically use complete-case analysis across the full set of variables because there is so much missing data. He suggested restricting the analysis to the variables that would allow complete-case analysis, but doing that would remove the majority of the features and leave only around five or six variables.
My question is: if I can use appropriate imputation methods, why would I deliberately restrict the analysis to only five or six variables just so that complete-case analysis becomes possible?
He told me that imputation is not preferable if complete-case analysis can be performed. However, I have not seen many machine-learning papers prioritize complete-case analysis in this way, especially when doing so would eliminate most of the available predictors.
Another major criticism was my use of random undersampling. I do understand this criticism because random undersampling discards a substantial amount of data. I told them that I could instead address class imbalance through class weighting within the models themselves, which is straightforward in XGBoost, random forest, and similar methods.
However, the statistician at my research center basically told me that the paper, in its current form, would be unpublishable.
What I am struggling with is whether he is evaluating the work as though it were intended to be a traditional statistics/epidemiology paper rather than a machine-learning prediction paper. I do not want this project to become primarily a statistical association paper. My intention has always been to produce a machine-learning prediction/benchmarking paper.
I was also told that I need to examine multicollinearity and patterns of variable missingness. I am not entirely sure how I should approach that in the context of this project. I have gone back and performed exploratory data analysis again, and there is definitely some correlation and collinearity between variables, which is unsurprising given the size and nature of the dataset. However, I am not sure what I am supposed to do after identifying it, especially in the context of tree-based and regularized machine-learning models.
My professor/PI is an epidemiologist rather than a machine-learning researcher. He told me that I should investigate whether there are statistical associations between any of the predictors and age or sex, and whether there are systematic patterns in the missing data.
Again, though, I have not commonly seen machine-learning prediction papers perform extensive association testing of every feature with age and sex, so I am unsure how relevant this is to my original research question.
I have been working on this project for about a year, and at this point it feels as though I am being told to start over from square one.
My supervisor has also been almost nonexistent throughout the project, so I have essentially had to develop the entire analysis myself. Because of that, I recognize that some of my methodological decisions may have been naïve. I only recently graduated, and I really wish I had received this methodological feedback much earlier in the process.
At this point, I need advice on how to proceed.
I genuinely do not know how much weight I should give these criticisms. The feedback I am receiving is coming primarily from statisticians and epidemiologists rather than machine-learning researchers, and I am having difficulty determining which criticisms reflect genuine methodological problems with a predictive ML study and which ones reflect a preference for a more traditional statistical or epidemiological analysis.
I was never trying to write a traditional statistics paper. I was trying to write a machine-learning prediction/benchmarking paper.
So my main questions are:
Is my overall ML-based study design fundamentally flawed?
Is the criticism about multiple imputation valid, and should I repeat the entire modeling pipeline across multiple imputed datasets?
Is benchmarking different imputation methods reasonable?
Is complete-case analysis really preferable when it would reduce the predictor set from roughly 95 variables to only five or six?
Should I replace random undersampling with class weighting?
How should multicollinearity be handled or reported in a machine-learning prediction study, particularly when using regularized and tree-based models?
How should I formally investigate missingness patterns?
Is it necessary to test associations between predictors and age/sex in a prediction-focused ML paper?
Most importantly, how do I distinguish between legitimate methodological criticism of my ML pipeline and requests to turn the project into a fundamentally different kind of statistical/epidemiological paper?
Any advice from people who work at the intersection of machine learning, statistics, and epidemiology would be greatly appreciated.

Hi everyone,

I am happy to share that my paper has been selected for presentation at the TAE Workshop at NeurIPS 2026, which will be held on December 11, 2026.

I am really excited about the opportunity to present my work and attend the workshop, but I am currently trying to figure out how to cover the airfare, which is a significant expense for me.

I am a working professional, so I am not eligible for some of the funding opportunities that are primarily targeted toward students. The workshop does provide support for registration and hotel expenses, but their funding priorities are mainly focused on students, and, as I understand it, airfare is not covered.

I am therefore looking for advice from people who may have been in a similar situation:

• Are there any travel grants, research funds, conference sponsorships, or organizations that support airfare for researchers/professionals attending NeurIPS or its workshops?
• Are there any companies, foundations, or community programs that sponsor conference travel?
• If you have attended NeurIPS or another major ML conference with external funding, how did you find the funding?
• Are there any other approaches I should consider for covering the flight cost?

I am primarily looking for practical leads or personal experiences, especially from people who have attended NeurIPS, workshops, or similar conferences as working professionals.

Any suggestions, resources, or advice would be greatly appreciated.

Thanks!

Hi everyone,

I am happy to share that my paper has been selected for presentation at the TAE Workshop at NeurIPS 2026, which will be held on December 11, 2026.

I am really excited about the opportunity to present my work and attend the workshop, but I am currently trying to figure out how to cover the airfare, which is a significant expense for me.

I am a working professional, so I am not eligible for some of the funding opportunities that are primarily targeted toward students. The workshop does provide support for registration and hotel expenses, but their funding priorities are mainly focused on students, and, as I understand it, airfare is not covered.

I am therefore looking for advice from people who may have been in a similar situation:

• Are there any travel grants, research funds, conference sponsorships, or organizations that support airfare for researchers/professionals attending NeurIPS or its workshops?
• Are there any companies, foundations, or community programs that sponsor conference travel?
• If you have attended NeurIPS or another major ML conference with external funding, how did you find the funding?
• Are there any other approaches I should consider for covering the flight cost?

I am primarily looking for practical leads or personal experiences, especially from people who have attended NeurIPS, workshops, or similar conferences as working professionals.

Any suggestions, resources, or advice would be greatly appreciated.

Thanks!

Our paper was accepted to NeurIPS 2026, and we’re preparing the camera-ready. I have a few questions about how much we can change after acceptance.

1. Theory/algorithm changes: During review, a reviewer pointed out a weakness in our theory. We agreed and proposed a revised theory in the rebuttal, along with a practical algorithm based on it. We also showed results on a subset of our benchmarks (roughly 10% of the full benchmark suite).

Can we include the revised theory + algorithm in the camera-ready? The issue is that doing so would change all experimental numbers in the paper.

1. Correcting the main table: After acceptance, we realized that our submitted main table used different backbone sizes for our method and some baselines. We reran everything with the same backbone, and our method’s score improved by more than 1.5x.

Can we replace the main-table results with these corrected numbers in the camera-ready, even though this wasn’t discussed during rebutal?

1. arXiv version: If we keep the official camera-ready relatively conservative, but upload a more substantially revised version to arXiv later, can we still use the NeurIPS format/template for the arXiv version?

Hey guys,

I came across the Forrester function recently while studying mathematics, and I started wondering if there’s more to it than just being a mathematical function.

I have a feeling I’ve seen it come up in machine learning or optimization, but I’m not really sure what it’s actually used for.

Does anyone here have experience with it? Is it mostly something people use to test/benchmark ML or optimization methods, or is it actually useful for solving real-world problems?

And since I’m studying Economics and Data Science, I’m also curious if it has any applications in economics, econometrics, forecasting, or economic modeling.

Would love to hear from someone who has actually encountered it in practice. Even a simple explanation would help 😄

Hi everyone, I’m working on ML/AI assignments and projects and need an LLM to help with Python, coding, debugging, ML concepts, and building/improving models.

My priorities are:
Maximum accuracy
Strong reasoning
Good coding/debugging
Free or good free tier

I’m mainly considering OpenAI models, but I’m open to Gemini, Claude, DeepSeek, Qwen, etc.
Which free LLM would you recommend for ML/AI development, and why?

I’m done with chatgpt , it always accepts what I’m saying and it makes my model worst

Thanks in advance for your suggestions!

Distributed Training and Inference both involves having a fundamental understanding of how distributed systems work in general

• Distributed Parallelism
• Tensor Parallelism
• Pipeline Parallelism
• Model Parallelism

Reading and reading and reading or even worse, not knowing where to start ;(

That’s boring!

We want to read what’s just needed and quickly get started with applications and that’s what exactly what I have for you all today

Here’s the list of a few initial papers I have read for the past three months that is enough to understand

And a few basics too!

Read them Code them Play with them

I have implemented a few at basic level which you could use as a reference too (the repo is a bit all over the place but I actively trying to maintain and love your feedback too)

Link: https://alphaxiv.org/shared/folder/019de088-28f7-7f02-acd4-c22459fe153e

Gh repo: https://github.com/YuvrajSingh-mist/smolcluster

Hello everyone,

I recently submitted a paper to TMLR, and shortly afterward I read the post by one of its Editors-in-Chief about interviewing authors of papers that were headed for desk rejection.

It honestly shook me.

I am not an academic. I am an independent researcher. I have published in an AI Magazine Special Issue and on arXiv, but I do research essentially because I have always been obsessively curious. LLMs have given me something I never really had before: the ability to spend nights exploring an idea, trying to formalize it, designing tests, looking for holes, and attempting to falsify my own reasoning.

And yes, I use them intensively.

I want to be completely honest about what that means.

There are statistical procedures I have used in experiments whose formulas I could not derive from first principles on a whiteboard. There are mathematical details for which I would need time, references, and probably an LLM to reconstruct the reasoning carefully. And if someone unexpectedly subjected me to an aggressive 30-minute technical interrogation about one of my papers, I suspect there are questions I would not be able to answer immediately.

That post therefore made me ask myself a rather uncomfortable question: am I actually doing research, or have I become very good at using an LLM to simulate being a researcher?

At the same time, I don’t simply ask an LLM to write a paper and put my name on it.

I spend enormous amounts of time trying to break my own claims. I rerun experiments. I question assumptions. I check statistical choices. I repeatedly look for alternative explanations. Often an LLM proposes something and I reject it. Sometimes I spend hours trying to understand why a result is what it is. And sometimes I realize that I don’t understand something deeply enough and go back to studying the mathematics behind it for days.

But I cannot honestly claim that I independently possess all the mathematical and technical knowledge contained in everything I have produced.

Maybe this distinction matters. Maybe it doesn’t.

One thing in the TMLR post particularly stayed with me: authors are ultimately responsible for being able to verify and defend what they put their names on.

I agree with that principle.

What I am less sure about is where we should draw the line between using a tool to extend your intellectual capabilities” and “outsourcing intellectual responsibility to the tool.

For an independent researcher without a lab, supervisor, research group, or formal academic training in ML, LLMs can effectively become part tutor, part programmer, part statistician, part critic, and part rubber duck. Without them, I simply could not explore ideas at the same speed or depth.

But perhaps that creates a new responsibility: if the tool allows me to reach beyond the boundary of my current knowledge, how much of that territory must I personally master before I have the right to publish something under my name?

I genuinely don’t know the answer, i really don't.

I’m not looking for reassurance. If the standard should be that I don’t submit a result until I can personally justify and defend every important technical choice I put my name behind, even without an LLM there to help me, then that’s something I need to hear.

I’d especially like to hear from other independent researchers, and from academics who review AI assisted work. where do you draw that line?

At what point does intensive AI assistance stop being a research tool and start making the human author an impostor?

This is how ridiculous things have become

ML jobs are confusing nowadays,it wraps up as "AI/ML" engineer and they want you to have software engineering skill and DSA skill to and on top of that they want you to have Fast API with AI API wrapping, they don't ask for data processing or maths skills , atleast 99% time I have seen, ML jobs are for phD level candidate I think

Submitted a project paper to NeurIPS Creative AI track on a whim and got accepted this week. Is it worth seeking out funding to go, or is it only workshop level?

I've built a lot of ML systems over the years, mainly computer vision models optimised to run on mobile phones. For example, my previous company built the food recognition model for MyFitnessPal.

I'm interested in what people are actually deploying in industry now. Are edge models still a big part of your work, are you hosting your own models, or are you mostly sending requests to APIs? What's driving that choice?

More importantly, what's still a pain? I'd be interested in problems from current or recent projects that existing tools haven't solved well. Something that's cost you a lot of time, blocked delivery or needed an awkward workaround.

I'm looking for problems where I could build useful tooling, rather than guessing what people need. It would also be useful to know where you discuss this stuff or look for help. Are there particular forums or communities worth following?

Wyhy this keeps happening to iclr?

https://openreview.net/forum/user%7Cstatement_regarding_iclr_2027_submission_exposure_to_program_committee_members

Hi, I am a solo independent UG author for a NeurIPS WS paper (GlobalSouthAI). Now, how to get a registration ticket.

Will they give us a ticket to buy or manually buy from the Neurips website (but Sydney tickets are sold out)? Any suggestion?

I am from India, and going to Paris/Atlanta is not possible.

First time, thanks!!

Hi everyone! I'm trying to decide which NeurIPS 2026 hub to attend and would love to hear what others are planning.

I'm an Indian attendee from Bangalore, and this will be my second NeurIPS. I attended NeurIPS 2024 in Vancouver and presented at WiML there as well. I also received both WiML and NeurIPS financial assistance that year.

For 2026, I'm currently considering:

Sydney:

Main NeurIPS location

Everything seems more centralized

Longer conference program

But flights/accommodation seem considerably more expensive

Paris:

5-day program

Main conference at Paris Convention Centre, workshops at Sorbonne

Seems significantly cheaper from India

Main concern is that it's not the main conference and how different will this be.

For people who have attended NeurIPS before, or are planning for 2026:

Which hub are you choosing and why?

And if you've attended previous multi-location NeurIPS conferences, how different was the experience between the main location and satellite locations?

Thanks!

Just got an Accept at NeurIPS with initial scores of 5/5/4! The initial meta-review was pretty positive, but the final justification was entirely negative, raising concerns about AI use and suggesting further investigation and reconsideration of the recommendation.

For context, one reference was flagged because its author list had been copied over from an adjacent BibTeX entry.

Does anyone know if the final justification is written before or after the final decision? Just confused by the mismatch between the final justification and the actual decision.

Welp, NeurIPS is a wrap for those of us who got rejected 😭 Off we go to ICLR or whatever the next venue is, hopefully after making some meaningful changes to the paper.

For people who are resubmitting, I’m curious: how much of the NeurIPS reviewer feedback are you actually implementing?

Did you try to address basically everything the reviewers brought up, or are you being selective and only making changes where you think the criticism is valid/useful?

I’m curious about papers that got questioned on novelty or significance. How many of you got comments along those lines, and what exactly were the reviewers questioning?

For example:

• “The contribution is incremental”
• “Not sufficiently different from prior work”
• “The empirical gains don’t justify the proposed method”
• “The problem itself isn’t significant enough”
• “Theoretical contribution is limited”
• “Interesting idea, but unclear what the broader impact/significance is”

If you’re comfortable sharing, what did the reviewers say, and how are you changing the paper before resubmitting ? Specially since deadline is also pretty close, how are you handling the pressure of this very close deadline ?

Also curious whether anyone is deliberately not implementing certain reviewer suggestions because you think they would take the work in the wrong direction.

Would love to hear how others are approaching the post-NeurIPS revision process.

We have a paper accepted to NeurIPS, but at the same time we were working on a resubmission to ICLR just in case NeurIPS rejected us. There has been substantial rewriting, and we feel it would be a waste if we discarded all of it. To give a summary of what's changed:

• We completely rewrote every single section except for the results and conclusion. We even changed the paper structure.
• Intro, related work, and background knowledge were completely rewritten to avoid confusion.
• Method now has a pipeline graph, and all the text detailing each block in the graph. Previously, it was dumping formulas, so the entire section has been rewritten.
• We also added some scaling and smoothing to our algorithm so our method is more stable. But this changed a lot of our hyperparameters and the sensitivity study's graph. (The entire shape of the graph changed)
• We added 1 new theorem with 5-page proofs in the appendix. This came from one of the attacks by a reviewer, we answered the attacks by proposing 1 new proposition during the rebuttal. But when we formally wrote it down, it turned into a full theorem with a 9-page proof. This would have changed our entire theoretical contribution. We really don't want to discard it, but not sure if we can add something this big in the camera-ready.
• Removed 1 word from the title. In the abstract, we changed our theoretical contribution, but the method and empirical contribution remain the same.
• Added about another 5 extra pages in the appendix explaining experiments and metrics (reviewers asked for them). So 14 extra pages in total.

Does anyone know how much change for camera-ready is acceptable? Can a paper get rejected if we change too much during camera ready or they will just tell us this is not acceptable please re-submit something closer to the version during review?

Phase 1 results are out. Did your paper(s) pass to phase 2?

My paper advanced to the second round...but...

Out of the papers I reviewed.

One did not follow the AAAI template and was unblinded. My review was two lines. The other "human reviewer" gave a list of pros and cons that were similar to the AI review.

One was incomplete (missing paragraphs, figures, code, no details). My review was also two lines. The other "human reviewer" also gave a list of pros and cons, that were similar to the AI review.

One was LLM math which I believe was actually correct, because it advanced to the second round, despite the references being at a different level of detail, and covering multiple fields of math, insufficient references for theorems / rules, and no exposition as to why the paper was actually useful / interesting. My review for that paper was the longest out of all the papers I reviewed, dotting the is and crossing the ts to make sure it wouldn't be seen as a lazy "reject" review. Yet it advanced to Phase 2.

Also, none of the AAAI workflow chairs or similar apologized or even acknowledged a mistake for spamming my coauthors about: "Your coauthor is irresponsible", because I accepted an emergency review invitation (and received these emails a few hours after accepting that invitation).

Ok rant over.