Difference between revisions of "Recursive wget"
Jump to navigation
Jump to search
| Line 1: | Line 1: | ||
'''How to wget something recursively without pulling lots of unnecessary crud''' | '''How to wget something recursively without pulling lots of unnecessary crud''' | ||
| + | |||
| + | -r means descend recursively | ||
| + | |||
| + | -np means never follow a link upward | ||
| + | |||
| + | -nH means don't put everything in a hostname subdir | ||
| + | |||
<pre> | <pre> | ||
wget -nH -r -np http://blah.blah.com/blah | wget -nH -r -np http://blah.blah.com/blah | ||
</pre> | </pre> | ||
| + | |||
| + | add -N to turn on timestamping, so files already stored locally will not be downloaded again unless changed. | ||
| + | |||
| + | add '-A suffix' to download only files with suffix 'suffix' | ||
| + | |||
| + | <pre> | ||
| + | wget -nH -r -np -A gif,jpg -N http://blah.blah.com/blah | ||
| + | </pre> | ||
| + | |||
[[Category:Incantations]] | [[Category:Incantations]] | ||
Latest revision as of 21:19, 12 November 2014
How to wget something recursively without pulling lots of unnecessary crud
-r means descend recursively
-np means never follow a link upward
-nH means don't put everything in a hostname subdir
wget -nH -r -np http://blah.blah.com/blah
add -N to turn on timestamping, so files already stored locally will not be downloaded again unless changed.
add '-A suffix' to download only files with suffix 'suffix'
wget -nH -r -np -A gif,jpg -N http://blah.blah.com/blah