Submit Your Article Forum Rules FAQ About Us
Search the forums:

Tech Support Team


Hello and Welcome to Tech Support Team! Before you can start posting and answering questions, you'll have to register. Registration is fast, simple and absolutely free! Feel free to browse through existing questions by choosing the forum you want to visit below.



Reply
  #1 (permalink)   Top
Old 31st July 2009, 06:44 PM
toarlach's Avatar
iEntry Staff
 
Join Date: Oct 2008, 42 posts.
Location: Richmond, KY, USA
Reputation: toarlach is on a distinguished road
Grabbing multiple files of a site using cURL in PHP

How would I use cURL in PHP to grab all of the html pages under a URL? I am
using this to grab all of the tables to append to a file in Linux (e.g. php
getfile.php | get table >> table.txt). This is what I have so far to cURL an
individual page...

HTML Code:
<?php
// create a new cURL resource
$ch = curl_init();

// set URL and other appropriate options
curl_setopt($ch, CURLOPT_URL, "http://www.fallingrain.com/world/AS/");
curl_setopt($ch, CURLOPT_HEADER, 0);

// grab URL and pass it to the browser
curl_exec($ch);

// close cURL resource, and free up system resources
curl_close($ch);

?> 
(yes, it's the cURL example off php.net)

I am also thinking of adding a sleep command for each iteration it uses to grab
the html files under the URL. This is so the site does not think I am attempting to attack it.

Can someone please steer me in the right direction of how would I do this? It is much appreciated.
__________________
Reply With Quote
Reply

Only registered members can participate in forum threads. You must register or log in to contribute.


Tags
curl, php

Thread Tools

Forum Jump


All times are GMT. The time now is 04:32 PM.






Post A Question!
Useful Links
Main Menu
Home
Forum Rules
FAQ
About Us
Welcome Pack
Search the forums
TST Mobile
Contact Us
Send Message

These are the 18 most used thread tags
Tag Cloud
32-bit cat drivers geforce hardware intel gfxui mobile 4 chipset driers modem monitor network no ring response no signal nvidia soft modem software wifi win7 windows 7