|
Are you Setting up your Screaming Frog Crawl Correctly?
When you first download Screaming Frog, there's one setting that can easily get missed, but it should be one of the first settings you change.
The first thing you should do is go to Configuration > System > Storage.
From there, change the Storage Mode from "Memory Storage" to "Database Storage".
This is easy to miss, but it can be really powerful.
It allows you to crawl sites with around 1 million URLs without any huge problems and should prevent most "out of memory" problems.
You'll only have to change this once, and it will also make it much easier for you to save your crawls.
|
