Task holds information to execute. This is non-spawned task structure as an object.
Members
AttempRule
# attempts Optional
retries or [retries, allowedStatuses, callback({ err, shouldRetry, task, crawler })].
Rule for plugin-attempt
true
# cancel Optional
Setting cancel to true will make crawler stop the execution of the task immediately.
FollowRule
# follow Optional
rule for plugin-follow: [selector, taskFactoryFunc, filter=urls=>urls.filter(v=>v)].
workflow:selector-parse => filterFunction => taskFactoryFunc for each
function
# parseCheck Optional
check before parsing if it exists, return true if you want to continue the parsing execution