NAME Parallel::Downloader - simple downloading of multiple files at once VERSION version 0.120101 SYNOPSIS use HTTP::Request::Common qw( GET POST ); use Parallel::Downloader 'async_download'; # simple example my @requests = map GET( "http://google.com" ), ( 1..15 ); my @responses = async_download( requests => \@requests ); # complex example my @complex_reqs = ( ( map POST( "http://google.com", [ type_id => $_ ] ), ( 1..60 ) ), ( map POST( "http://yahoo.com", [ type_id => $_ ] ), ( 1..60 ) ) ); my $downloader = Parallel::Downloader->new( requests => \@complex_reqs, workers => 50, conns_per_host => 12, aehttp_args => { timeout => 30, on_prepare => sub { print "download started ($AnyEvent::HTTP::ACTIVE / $AnyEvent::HTTP::MAX_PER_HOST)\n" } }, debug => 1, logger => sub { my ( $downloader, $message ) = @_; print "downloader sez [$message->{type}]: $message->{msg}\n"; }, ); my @complex_responses = $downloader->run; DESCRIPTION This is not a library to build a parallel downloader on top of. It is a downloading client build on top of AnyEvent::HTTP. Its goal is not to be better, faster, or smaller than anything else. Its goal is to provide the user with a single function they can call with a bunch of HTTP requests and which gives them the responses for them with as little fuss as possible and most importantly, without downloading them in sequence. It handles the busywork of grouping requests by hosts and limiting the amount of simultaneous requests per host, separate from capping the amount of overall connections. This allows the user to maximize their own connection without abusing remote hosts. Of course, there are facilities to customize the exact limits employed and to add logging and such; but "async_download" is the premier piece of API and should be enough for most uses. FUNCTIONS async_download Can be requested to be exported, will instantiate a Parallel::Downloader object with the given parameters, run it and return the results. METHODS run Runs the downloads for the given parameters and returns an array of array references, each containing the decoded contents, the headers and the HTTP::Request object. SUPPORT Bugs / Feature Requests Please report any bugs or feature requests through the issue tracker at . You will be notified automatically of any progress on your issue. Source Code This is open source software. The code repository is available for public review and contribution under the terms of the license. git clone https://github.com/wchristian/parallel-downloader.git AUTHOR Christian Walde COPYRIGHT AND LICENSE This software is Copyright (c) 2012 by Christian Walde. This is free software, licensed under: DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE, Version 2, December 2004