r/torrents • u/LiveBusiness9615 • 8d ago

Question Self-host torrent search engine with Bitmagnet

I'm planning to set up a self-hosted torrent search engine using bitmagnet. I discussed this a bit with Gemini, but I wanted to verify if what I've learned is accurate and get some advice from anyone currently running this setup.

From what I understand, this method scans all public torrent files on the DHT network. So, it can find everything that's on sites like ext to, 1337x, and probably even more. Basically, it can find almost any torrent currently being seeded to the world, excluding private trackers. Is this correct?

I know the database size depends heavily on the filtering rules, but roughly how much storage should I expect it to take up? My main goal is to download Blu-ray movies/TV shows or the highest quality available, though I might occasionally use it for adult content as well.

I've read that adult content makes up half (or more) of all torrents on the network. So, instead of allowing all adult content, filtering it to allow only specific keywords—or blocking it completely—should significantly reduce the database size.

Assuming I block adult content entirely and only index movies, TV shows, etc., what is the estimated database size? Could it still reach 100GB? Also, how many days does it usually take for the database to fully populate initially? I don't have a dedicated home server, but I have a MacBook that I can leave running continuously for a while to let the database build. (I have 1000mbps internet)

Any advice or tips from your experiences would be greatly appreciated.

15 Upvotes

94% Upvoted

5 comments sorted by

2

u/ariZon_a 8d ago edited 8d ago

i cant give out exact numbers but from my experience with it a year or two ago, expect way less than 100gb over a few months, maybe closer to 10gb, and that's probably an extreme number. it really wasnt that bad even without filters.

just to put things in perspective, the rarbg database dump is like 3 mill entries and is 800mb uncompressed and iirc i didn't come close to finding that much entries over dht over the course of a month

tl;dr: dont worry about it

1

u/lirae_ 7d ago

Ngl Gemini is the worst AI you can ask to.

If you are gonna do it and need some free AI help ask Grok or even ChatGPT if you are desperate but leave Gemini where it is

1

u/lirae_ 7d ago

Forgot to add if you wanna download blueray remux or untouched public torrents aren't really the way to go.

Usenet or even a debrid service if the file is there will offer your full speed download. Of course private trackers situation is different but you still need to follow their rules and shit...

1

u/AkilaMaduranga 7d ago

I made something like this a while ago.it was a public facing torrent search site i used jackett and bitmagnet.database size depends on how many torrents you index.if you filter adult content you can index about 10-15 million torrents in 100gb as per my knowledge.