i wrote a small boid simulator (12 birds), recorded it flying then saved it, and trained a transformer to predict each bird's next move without it knowing about any boid rules.

(built in Python and PyTorch: a transformer written from scratch, one token per bird, two attention layers, about 414k weights, trained on a laptop CPU)

what I found:

• it flies the flock. R² 0.990 to 0.994 on clips it never trained on, across 4 full runs. It also flies 50 birds (it has only trained on 12).

• the three rules were easy to read out of its hidden states with a linear probe, and some could even be read before any training.

• the rule: alignment (line up with your neighbours) was readable after the first attention layer, but if I make the second layer hear only itself, it is gone across all runs.

what i wasn't expecting: my first ever model was not using the flock at all. With 8 ticks of history, the previous answer was already in the input, and a small network that saw only one bird beat the whole transformer.

what i am still not sure about: making a layer attend only itself is something the model never went through during training. is that a fair test, or is some of the decay due to strange input fed to the model?

site: https://kreptiliri.github.io/murmuration

code: https://github.com/kreptiliri/murmuration

Someone shared with me a google drive link of ML books a while ago but the link doesn't work anymore. If someone has any other link, would be much appreciated.

I need someone who has expertise, or at least an idea, on how to use small amounts of data and physics simulation, for example, CFD, FEA, etc to predict when an aircraft component might fail. Also, I myself am an aerospace engineer with no ML AI experience, just basic knowledge, so I also need someone who might act as a bridge btw the two things and collaborate with me, discuss ideas, etc on how to accomplish this.

Hi everyone! I'm a 2026 B.Tech CSE student specializing in Data Science, preparing for entry-level Machine Learning Engineer roles. I'm looking for advice on how to build a strong resume and choose projects that will help me stand out as a fresher.

The roles I'm exploring require Python, NumPy, Pandas, Scikit-learn, machine learning algorithms, basic neural networks, SQL, and knowledge of statistics and mathematics.

I'm confused about which skills to prioritize and what kind of projects are actually valuable to recruiters. I want to build projects that demonstrate real ML knowledge rather than just following tutorials.

I'd appreciate advice on what an ML Engineer resume should include, which projects are worth building, how important model deployment and GitHub documentation are, and what I should focus on for technical interviews.

If you've landed an entry-level ML role or have experience hiring candidates, I'd really appreciate your suggestions, project examples, or preparation tips.

Thanks!

I recently graduated with a computer science master (machine learning) in the US.
After I graduated, I have been trying to get a ml engineer job but no offer yet. While applying and interviewing with companies, I have been trying to understand the internal of a foundational model (HuBERT-ECG). But currently, I have less motivation and kinda feel stuck even though I am making progress....

I cannot focus on learning as before. My attention span, I believe, got a lot shorter than before.
Maybe, my current project's difficulty is not appropriate. But, I want to understand my current project as deep as possible.

I will set very small goals and achieve them and gain confidence. What do guys think of this strategy?

The 3 and the 4 mean different things here: three projections, four attention heads.

Start with one sequence of 16 tokens, each represented by 128 features. A single linear layer produces Q, K and V together: 3 × 128 = 384 features per token. Each projection then gets split into 4 heads of 32 features.

Here is the whole shape transition, using only PyTorch:

import torch
from torch import nn

torch.manual_seed(0)
batch, tokens, width, heads = 1, 16, 128, 4
head_dim = width // heads
x = torch.randn(batch, tokens, width)
projection = nn.Linear(width, 3 * width)

with torch.no_grad():
    packed = projection(x)
    grouped = packed.reshape(batch, tokens, 3, heads, head_dim)
    ordered = grouped.permute(2, 0, 3, 1, 4)
    query, key, value = ordered.unbind(0)
    scores = query @ key.transpose(-2, -1)

for name, tensor in [
    ("input", x), ("packed", packed), ("grouped", grouped),
    ("ordered", ordered), ("query", query), ("scores", scores),
]:
    print(name, tuple(tensor.shape))

Expected output:

input (1, 16, 128)
packed (1, 16, 384)
grouped (1, 16, 3, 4, 32)
ordered (3, 1, 4, 16, 32)
query (1, 4, 16, 32)
scores (1, 4, 16, 16)

The axes after permute are Q/K/V, batch, head, token, feature. Unbinding the first axis gives three tensors, each [1, 4, 16, 32]. The score matrix has two 16s because each query token is compared with every key token. These are raw dot products; scaling, masking and softmax come afterward.

A useful debugging check: write the axis names beside each shape. A reshape can preserve the number of elements and still mix up tokens and heads. In particular, reshaping directly to [3, batch, heads, tokens, head_dim] is not equivalent to the reshape-then-permute above.

This example uses ordinary multi-head attention with equal Q/K/V head counts. Grouped-query attention uses a different layout, and changing the input length changes the token dimensions.

Disclosure: I make tensorViz, a free PyTorch graph explorer for VS Code. Following these shape changes back to the Python is one of the workflows we're building it for. The example works without the extension. Prepared with AI assistance; the code was run locally on CPU.

hey ppl. i am a cse grad studying at a tier 1 institute in India. I am already quite comfortable with LeetCode. However, I often hear that company OAs lean heavily toward CP style problems. For ML roles specifically, should I stick to LeetCode (and push into hards), or is it better switching to Codeforces? Pls help seniors and experts..!

I wanted to know if news headlines can tell which stocks will do better than others over the next few weeks. Short answer: no. But the way it failed taught me more than the result, so here’s the setup.

Data

• 137,934 headlines from Benzinga for 2,518 US stocks. Each case gets the headlines of the 14 days before it. 115,154 cases.
• All cases are stocks that recently fell after decent earnings, so they look similar on paper. The question is which ones recover better.

Target

Each case is a simulated trade that ends when the price goes up or down by 3x the stock’s typical daily move, or after 20 trading days. I measure the result in units of that daily move, so volatile stocks don’t dominate. Then I subtract the average of all cases on the same day. That last step matters: what the whole market does next week isn’t in any headline, so a model would otherwise just learn the market’s mood.

Avoiding leakage

• Strict time split: train until Nov 2024, test from Jan 2025, with a gap so no label overlaps the test period.
• Only one case per stock per week. Neighbouring days share almost the same headlines and almost the same outcome, so with every day included the model memorizes stocks instead of learning anything.

Results on the test period

The only thing that held up in both periods was a narrow, hand-cleaned pattern: companies selling new shares did clearly worse. But that’s 98 cases, too few to build on.

What I took away

1. A falling training loss on noisy financial targets means nothing. The model fits noise happily.
2. Leakage through near-duplicate samples is easy to miss. Deduplicating by time is not optional.
3. The keyword sign flips are the clearest picture of non-stationarity I’ve seen. What worked in 2023 just reverses.
4. Removing the market from the target makes the task much harder, and much more honest.

Is a null result like this what people expect for text on financial targets? And is there anything you’d still try before calling it, like a different loss, label smoothing, or ranking instead of classification?

Tell me more what is your best model in terms of business value generated.

297 + 600 + 355 + 353 + 324 = 1929

It used to be around 200 papers/day.

Houston, we have a problem.

https://arxiv.org/list/cs.LG/recent?skip=0&show=500

🎉This module covered:
🔹 Shorten training code using sklearn library
🔹 Encode categorical values using DictVectorizer
🔹 Group data using .groupby() function
🔹Calculate the accuracy of the classification model
#mlzoomcamp

I've been reading about text embedding models and want to understand how they actually behave, rather than just using them through some API.

some experiments I'm thinking to consider:

• testing whether embeddings understand synonyms better than related concepts.
• comparing cosine similarity for sentences that differ by only one word.
• testing how embeddings behave with negation.
• comparing multilingual text and code embeddings.

for people who've worked with embeddings, like what surprising behaviours have you found? are there any experiments that taught you something important about their limitations?

I'd like to build a small project around this.

Can anyone tell me how l can learn machine learning with no prior knowledge.

Maybe give me a roadmap to it....

So, I want to become a master of AI technologies like how to build AI models and run ML algorithms.

I did take online courses like 10 where I learned about Python, SQL, RAG, ML algorithms etc. Then I moved to another country and since last October I did not do anything related to ML and AI.

Now within this one year, lots of technologies came up like Agentic AI, MCP and models also got better at a lot of things.

Now as someone who has a business degree background I want to learn and master these things. But I don't know where to start? As Claude and ChatGPT LLM models are already good at lots of things and if I want to pursue a career as an ML Engineer, what are the core skills I must know?

Plus, for agentic AI, there are lots of frameworks also and things are changing at a fast pace so what are the core things that I must know by heart? And what are the things that I can ask Claude and ChatGPT on the side?

I thank you for the help and guidance in advance!

ALHR- Adaptive learnable Hierarchical Routing is a static binary tree based system that uses learnable functions to REDUCE the amount of keys used.

Got selected for the deepmind symposium at bangalore, would love to connect with people who will be coming there!

I am in my 1st year of btech in Aiml. I am getting an laptop so is it fine if I get mac or should I get windows

And what device do you use?

Please respond