#
Explaining threads
Think of a thread as a worker. In a computer program, this worker is given a list of tasks to do.
#
One Thread vs. Many Threads
Imagine you're running a bakery. If you have only one worker (single-threading), that worker has to do everything - bake the bread, serve the customers, clean the shop. If the worker is busy with one task, everything else has to wait.
But what if you could have multiple workers (multi-threading)? One could bake the bread, another could serve the customers, and a third could clean the shop. They can all work at the same time, so things get done faster!
#
Threads and Versatile
Imagine you're running a big bakery and you have a huge order to fill. You need to bake thousands of loaves of bread by the end of the day. Would you rather have one worker doing all the baking, or hundreds of workers each baking a few loaves?
In web scraping, each "loaf of bread" is a webpage you want to scrape, and each "worker" is a thread. If you use more threads, you can scrape more webpages at the same time, just like having more workers lets you bake more loaves at the same time.
#
How many threads should I use?
Just like a bakery can only fit so many workers, your computer can only handle so many threads. If you try to use too many threads, your computer might slow down, or your program might crash. That's why we recommend using between 50 and 4000 threads for Versatile, depending on how powerful your computer is.
Most people find that 500 threads is a good balance between speed and stability. It's like having just the right number of workers in your bakery - not so few that you can't fill your order, but not so many that they get in each other's way.