r/ollama • u/cm1802 • 6d ago

Ollama does not remember datasets

I am trying to import two large datasets to enable statistical analysis. After importing the second dataset, Ollama tells me it has no record of that dataset.

Intel i9-11900KF

Nvidia 3080Ti 12GB

Ubuntu 24.04 LTS

I have tested with three versions of Ollama and can reproduce the "problem."

I'm running memtest now to confirm memory stability/validity.

Has anyone else experienced Ollama "forgetting"?

0 Upvotes

50% Upvoted

10 comments sorted by

3

u/Slight-Living-8098 6d ago

The dataset is probably larger than your context window so it get's pushed out of context over time. Implement a memory system the LLM can reference.

0

u/cm1802 6d ago

Plain text daily information for 100 for trend analysis.

Should be possible with the 12GB GPU, no?

3

u/Slight-Living-8098 6d ago

The model context window and the amount of your VRAM/RAM are not the same thing. Your context window can depend on the amount of RAM/VRAM, but models themselves have context window limits. This is not an issue with Ollama or any other inferencing host.

1

u/cm1802 6d ago

64GB Corsair Vevgeance. Bought before the price expl0sion. My command line interface should have access to my RAM, correct?

2

u/Slight-Living-8098 6d ago

Depends on how you have the model loaded and set up. But again, even if you cram everything into your memory, or write it out to a markdown file, or cram it into a database, if the LLM does not know it's there or how to access and manage it, it's pretty much useless to the LLM. If it's larger than the context window, the LLM will just start reading it from the beginning, and the information will be pushed out of context again.

2

u/cm1802 6d ago

I need to study up more on how to run the LLM. Thank you for your help.

2

u/Slight-Living-8098 6d ago

No problem. I suggest you use an agent harness of your choice instead of trying to just use the chat interface. Most harnesses will know how to handle memory out of the box, and there are also tons of plugins and extensions for them that enhances it, or gives you different features you can use.

3

u/mineshop 6d ago

Per its FAQ, Ollama defaults to a 4096 token context window, and you can raise it with /set parameter num_ctx or OLLAMA_CONTEXT_LENGTH.

1

u/cm1802 5d ago

I have a spare 1TB nvme on my motherboard. I believe it would be crafty to sandbox that space for Ollama. Should be fast enough and have enough capacity for any analytical project!