How would I use cURL in PHP to grab all of the html pages under a URL? I am
using this to grab all of the tables to append to a file in Linux (e.g. php
getfile.php | get table >> table.txt). This is what I have so far to cURL an
individual page...
HTML Code:
<?php
// create a new cURL resource
$ch = curl_init();
// set URL and other appropriate options
curl_setopt($ch, CURLOPT_URL, "http://www.fallingrain.com/world/AS/");
curl_setopt($ch, CURLOPT_HEADER, 0);
// grab URL and pass it to the browser
curl_exec($ch);
// close cURL resource, and free up system resources
curl_close($ch);
?>
(yes, it's the cURL example off php.net)
I am also thinking of adding a sleep command for each iteration it uses to grab
the html files under the URL. This is so the site does not think I am attempting to attack it.
Can someone please steer me in the right direction of how would I do this? It is much appreciated.