The most common action that a crawler takes is to follow links on a page. This module helps you with that task. WWW: https://metacpan.org/release/GunghoX-FollowLinks