Home » HTTP » HTTP_Request2 » Manual

Introduction

Introduction – Introduction to HTTP_Request2

Package Overview

HTTP_Request2 package provides an easy way for PHP applications to perform HTTP requests. It supports a large subset of Hypertext Transfer Protocol features, and can be used for the following:

• Working with web services (numerous PEAR packages in Web Services category are using HTTP_Request2 under the hood);
• Checking the validity of web links;
• Grabbing and parsing remote web pages;
• Automated form submission.

Basic Usage Example

Performing a request with HTTP_Request2 consists of the following steps

• Creating, configuring and populating an instance of HTTP_Request2 class. At the very least you should set request URL and maybe proxy parameters (if you are using proxy).
• Calling send method of that instance. This will pass control to an Adapter that will send the request and read remote server's response. Request's progress may be monitored by Observers.
• Processing the returned instance of HTTP_Request2_Response. An instance of HTTP_Request2_Exception can also be thrown by send() if response could not be received (completely or at all) or parsed.

Fetches and displays PEAR website homepage

<?php
require_once 'HTTP/Request2.php';

$request = new HTTP_Request2('http://pear.php.net/', HTTP_Request2::METHOD_GET);
try {
    $response = $request->send();
    if (200 == $response->getStatus()) {
        echo $response->getBody();
    } else {
        echo 'Unexpected HTTP status: ' . $response->getStatus() . ' ' .
             $response->getReasonPhrase();
    }
} catch (HTTP_Request2_Exception $e) {
    echo 'Error: ' . $e->getMessage();
}
?>

Configuration

Configuration – Configuration parameters for HTTP_Request2

List of Configuration Parameters

HTTP_Request2 constructor and HTTP_Request2::setConfig() method accept the following configuration parameters. You can also use HTTP_Request2::getConfig() to get the values of these parameters. Note that using an unknown parameter name will result in an exception.

Proxy Configuration

While most of the configuration parameters have sane default values and generally can be left alone, you'll definitely need to configure proxy you are using to access websites.

Proxy configuration example

<?php
require_once 'HTTP/Request2.php';

$request = new HTTP_Request2();
$request->setConfig(array(
    'proxy_host'        => 'proxy.example.com',
    'proxy_port'        => 3128,
    'proxy_user'        => 'luser',
    'proxy_password'    => 'sekret',
    'proxy_auth_scheme' => HTTP_Request2::AUTH_DIGEST
));

// Now you can perform requests
?>

SSL peer verification issues

For SSL peer verification to work, OpenSSL library (used under the hood by both Curl and Socket adapter) needs certificate authority files. However it does not include any such files itself, expecting OS distributions compiling OpenSSL library to provide proper default locations.

Unfortunately OpenSSL extension of PHP below version 5.6 does not try to use the distribution-default values for CA file / CA path when explicit ones are not provided. Curl extension, however, does use these defaults, so you can sometimes be able to use 'verify_peer' without setting 'ssl_cafile' for Curl adapter, but not for Socket one (see e.g. bug #18480 and bug #19351).

For versions of PHP below 5.6 the only solution is to provide 'ssl_cafile' and / or 'ssl_capath' for every request if using Socket adapter. Curl adapter will or will not be able to use the defaults depending on distribution, additionally you can set curl.cainfo parameter in php.ini on PHP 5.3.7+ (it contains the default value for CURLOPT_CAINFO setting to which 'ssl_cafile' setting is mapped).

PHP version 5.6 will finally enable peer verification by default for 'ssl' stream wrapper, so it will also provide more possibilities for using default values:

• PHP will use defaults compiled into the OpenSSL library by distribution.
• openssl.cafile and openssl.capath settings in php.ini will provide default values for 'cafile' and 'capath' SSL stream context options ('ssl_cafile' and 'ssl_capath' settings map to these in Socket adapter).

HTTP_Request2

HTTP_Request2 – Class representing a HTTP request message

Request URL and GET Parameters

Request URL can be set in HTTP_Request2 constructor or via HTTP_Request2::setUrl() method. Both of these accept either a string or an instance of Net_URL2. URL is stored internally as an instance of Net_URL2 that can be accessed via HTTP_Request2::getUrl().

GET request parameters can be added to URL via Net_URL2::setQueryVariable() and Net_URL2::setQueryVariables():

Setting GET parameters

<?php
// Explicitly set request method and use_brackets
$request = new HTTP_Request2('http://pear.php.net/bugs/search.php',
                             HTTP_Request2::METHOD_GET, array('use_brackets' => true));
$url = $request->getUrl();
$url->setQueryVariables(array(
    'package_name' => array('HTTP_Request2', 'Net_URL2'),
    'status'       => 'Open'
));
$url->setQueryVariable('cmd', 'display');

// This will output a page with open bugs for Net_URL2 and HTTP_Request2
echo $request->send()->getBody();
?>

HTTP Authentication

HTTP_Request2 supports both Basic and Digest authentication schemes defined in RFC 2617. Authentication credentials can be set via HTTP_Request2::setAuth() method or given in the request URL (but in the latter case authentication scheme will default to Basic).

Setting authentication credentials

<?php
// This will set credentials for basic auth
$request = new HTTP_Request2('http://user:password@www.example.com/secret/');

// This will set credentials for Digest auth
$request->setAuth('user', 'password', HTTP_Request2::AUTH_DIGEST);
?>

There is currently an issue with Digest authentication support in Curl Adapter due to an underlying PHP cURL extension problem.

Request Headers

Additional request headers can be set via HTTP_Request2::setHeader() method. It also allows removing previosly set headers

Setting request headers

<?php
// setting one header
$request->setHeader('Accept-Charset', 'utf-25');

// setting several headers in one go
$request->setHeader(array(
    'Connection' => 'close',
    'Referer'    => 'http://localhost/'
));

// removing a header
$request->setHeader('User-Agent', null);
?>

Cookies

Cookies can be added to the request via setHeader() method, but a specialized HTTP_Request2::addCookie() method is also provided

Adding cookies to the request

<?php
$request->addCookie('CUSTOMER', 'WILE_E_COYOTE');
$request->addCookie('PART_NUMBER', 'ROCKET_LAUNCHER_0001');
?>

Using cookie jar to manage cookies across requests

Since release 2.0.0beta1 the package contains HTTP_Request2_CookieJar class that can be used manage cookies across requests. You can enable this functionality by passing either an existing instance of HTTP_Request2_CookieJar or TRUE to create a new instance to HTTP_Request2::setCookieJar() method. Cookie Jar can be later accessed by HTTP_Request2::getCookieJar().

Cookie jar will automatically store cookies set in HTTP response and pass them to requests with URLs matching cookie parameters. You can also manually add cookies to jar by using HTTP_Request2_CookieJar::store() method.

By default Public Suffix List is used to check whether a cookie domain matches the given URL. The same list is used to restrict cookie setting by Firefox, Chrome and Opera browsers. It can be disabled if needed by HTTP_Request2_CookieJar::usePublicSuffixList().

HTTP_Request2_CookieJar implements Serializable interface and thus can be easily serialized and stored somewhere. You can control whether session cookies stored in a jar should be serialized by calling HTTP_Request2_CookieJar::serializeSessionCookies().

When HTTP_Request2 instance has a cookie jar set, HTTP_Request2::addCookie() method will add a cookie to jar, rather than directly to 'Cookie' header, using current request URL for setting its 'domain' and 'path' components.

Request Body

If you are doing a POST request with Content-Type 'application/x-www-form-urlencoded' or 'multipart/form-data' (in other words, emulating POST form submission), you can add parameters via HTTP_Request2::addPostParameter() and file uploads via HTTP_Request2::addUpload(). HTTP_Request2 will take care of generating proper request body. File uploads will be streamed from disk by HTTP_Request2_MultipartBody to reduce memory consumption.

Emulating POST form submission

<?php
$request = new HTTP_Request2('http://www.example.com/profile.php');
$request->setMethod(HTTP_Request2::METHOD_POST)
    ->addPostParameter('username', 'vassily')
    ->addPostParameter(array(
        'email' => 'vassily.pupkin@mail.ru',
        'phone' => '+7 (495) 123-45-67'
    ))
    ->addUpload('avatar', './exploit.exe', 'me_and_my_cat.jpg', 'image/jpeg');
?>

addUpload() can accept either a string with a local file name or a pointer to an open file, as returned by fopen(). You currently can't directly pass a string with the file contents, however you can pass a pointer to php://memory or php://temp:

<?php
$fp = fopen('php://temp/maxmemory:1048576', 'r+');
fwrite($fp, generateFileUploadData());
$request->addUpload('stuff', $fp, 'custom.name', 'application/octet-stream');
?>

HTTP request body can also be set directly, by providing a string or a filename to HTTP_Request2::setBody() method. This is the only way to set a request body for non-POST request.

Setting "raw" request body

<?php
$request = new HTTP_Request2('http://rpc.example.com');
$request->setMethod(HTTP_Request2::METHOD_POST)
    ->setHeader('Content-type: text/xml; charset=utf-8')
    ->setBody(
        "<?xml version=\"1.0\" encoding=\"utf-8\"?" . ">\r\n" .
        "<methodCall>\r\n" .
        " <methodName>foo.bar</methodName>\r\n" .
        " <params>\r\n" .
        "  <param><value><string>Hello, world!</string></value></param>\r\n" .
        "  <param><value><int>42</int></value></param>\r\n" .
        " </params>\r\n" .
        "</methodCall>"
    );
?>

HTTP Redirects

Since release 0.5.0 HTTP_Request2 can automatically follow HTTP redirects if follow_redirects parameter is set to TRUE.

HTTP_Request2 will only follow redirects to HTTP(S) URLs, redirects to other protocols will result in an Exception.

HTTP_Request2::send will return only the final response, if you are interested in the intermediate ones you should use Observers.

Adapters

Adapters – Classes that actually perform the request

Overview

Adapters in HTTP_Request2 package are classes responsible for establishing the actual connection to the remote server, writing requests and reading responses. Adapter can be set either via a configuration parameter (see: Configuration) or by HTTP_Request2::setAdapter() method.

The package currently contains three adapters:

HTTP_Request2_Adapter_Socket

This Adapter uses PHP's builtin stream_socket_client() function and does not require any special extensions or configuration. Note that this is a pure PHP implementation of HTTP protocol, it does not use http stream wrapper.

This is the default Adapter, it will be used by HTTP_Request2 unless you explicitly set the other one.

HTTP_Request2_Adapter_Curl

Curl adapter wraps around PHP's cURL extension and thus allows using extremely sophisticated cURL library. It is a good choice if you have cURL extension available in PHP.

HTTP_Request2_Adapter_Mock

This adapter allows testing packages and applications using HTTP_Request2 without depending on a network connection and remote server availability.

You can replace the default adapter in HTTP_Request2 by an instance of HTTP_Request2_Adapter_Mock populated via its HTTP_Request2_Adapter_Mock::addResponse() method. The adapter will return prepared responses or throw prepared exceptions when its sendRequest() method is called.

Other adapters may be created by subclassing an abstract HTTP_Request2_Adapter class and implementing its HTTP_Request2_Adapter::sendRequest() method.

Socket Adapter Issues

Due to PHP bug #47030 ssl_verify_host configuration parameter is useless without ssl_verify_peer. When the latter is switched off no host validation will be performed.

Curl Adapter Issues

When doing a POST request with file uploads or reading the request body from file, HTTP_Request2 streams files from disk to reduce memory consumption and to allow monitoring the request progress. This is done by setting up a CURLOPT_READFUNCTION callback. PHP does not allow setting another callback, CURLOPT_IOCTLFUNCTION (see PHP bug #47204) so the request body can not be "rewound" when another request should be performed. Thus a request with a non-empty body to a resource protected by Digest authentication or to a page that does a redirect when follow_redirects is enabled will fail.

Since release 0.5.0 HTTP_Request2 works around this problem by reading the whole request body into memory. Of course this may be an issue when the request body is huge, so consider using Socket Adapter.

Setting strict_redirects configuration parameter to TRUE will only have effect on Curl Adapter if CURLOPT_POSTREDIR (see PHP bug #49571) is available in cURL extension. It is available in PHP 5.3.2 and higher.

Using Mock adapter

HTTP_Request2_Adapter_Mock::addResponse() method can accept either a HTTP response (instance of HTTP_Request2_Response, a string, a pointer to an open file) or an Exception. The latter may be useful to test that your application degrades gracefully when the remote server is not available due to some network problems. Several responses may be added via addResponse() to simulate a HTTP transaction consisting of multiple requests and responses. They will be returned by the Adapter in the order they were added.

Since release 2.1.0 it is possible to specify a request URL to addResponse(). sendRequest() will only return responses without an explicit URL set or having the same URL as the request. It is recommended that you set explicit URLs either for all responses or for none of them, otherwise the behaviour will be difficult to predict.

Returning responses and throwing exceptions

<?php
require_once 'HTTP/Request2.php';
require_once 'HTTP/Request2/Adapter/Mock.php';

$mock = new HTTP_Request2_Adapter_Mock();
$mock->addResponse(
    "HTTP/1.1 200 OK\r\n" .
    "Connection: close\r\n" .
    "\r\n" .
    "Explicit response for example.org",
    'http://example.org/'
);
$mock->addResponse(
    "HTTP/1.1 200 OK\r\n" .
    "Content-Length: 32\r\n" .
    "Connection: close\r\n" .
    "\r\n" .
    "Nothing to see here, move along."
);
$mock->addResponse(
    new HTTP_Request2_Exception("The server is on fire!")
);

// this request will succeed
$request1 = new HTTP_Request2(
    'http://ok.example.com/', HTTP_Request2::METHOD_GET,
    array('adapter' => $mock)
);
echo "Response: " . $request1->send()->getBody() . "\r\n";

// this request will fail
$request2 = new HTTP_Request2('http://burning.example.com/');
$request2->setAdapter($mock);
try {
    echo "Response: " . $request2->send()->getBody() . "\r\n";
} catch (Exception $e) {
    echo "Error: " . $e->getMessage() . "\r\n";
}

// The response for example.org was ignored by previous requests
$requestOrg = new HTTP_Request2(
    'http://example.org/', HTTP_Request2::METHOD_GET,
    array('adapter' => $mock)
);
echo "Response: " . $requestOrg->send()->getBody() . "\r\n";
?>

The above code will output:

Response: Nothing to see here, move along.
Error: The server is on fire!
Response: Explicit response for example.org

    

Mock Adapter also has two static helper methods HTTP_Request2_Adapter_Mock::createResponseFromString() and HTTP_Request2_Adapter_Mock::createResponseFromFile() which build HTTP_Request2_Response objects from either a string containing the complete HTTP response or a pointer to an open file with such response, respectively.

HTTP_Request2_Response

HTTP_Request2_Response – Class representing a HTTP response message

Overview

HTTP_Request2_Response encapsulates a HTTP response message and provides easy access to different parts of it. It also contains static helper methods HTTP_Request2_Response::decodeGzip() and HTTP_Request2_Response::decodeDeflate() for decoding response bodies encoded by Content-Encoding: gzip (as defined in RFC 1952) and Content-Encoding: deflate (RFC 1950), respectively.

An instance of this class will usually be returned by HTTP_Request2::send() method. You can also build an instance yourself using either helper methods of HTTP_Request2_Adapter_Mock or manually (see below).

You can use the following methods for accessing various parts of HTTP response: getStatus(), getReasonPhrase(), getVersion(), getHeader(), getCookies(), getBody(). Note that the response object will not contain the response body if 'store_body' configuration parameter was set to FALSE. Also note that getBody() always returns body completely decoded, if for some reason you want to access body still encoded by gzip / deflate you'll need to use Observers and Socket adapter.

Accessing response parts

<?php
$request  = new HTTP_Request2('http://www.example.com/');
$response = $request->send();

echo "Response status: " . $response->getStatus() . "\n";
echo "Human-readable reason phrase: " . $response->getReasonPhrase() . "\n";
echo "Response HTTP version: " . $response->getVersion() . "\n";
echo "Response headers:\n";
foreach ($response->getHeader() as $k => $v) {
    echo "\t{$k}: {$v}\n";
}
echo "Value of a specific header (Content-Type): " . $response->getHeader('content-type') . "\n";
echo "Cookies set in response:\n";
foreach ($response->getCookies() as $c) {
    echo "\tname: {$c['name']}, value: {$c['value']}" .
         (empty($c['expires'])? '': ", expires: {$c['expires']}") .
         (empty($c['domain'])? '': ", domain: {$c['domain']}") .
         (empty($c['path'])? '': ", path: {$c['path']}") .
         ", secure: " . ($c['secure']? 'yes': 'no') . "\n";
    
}
echo "Response body:\n" . $response->getBody();
?>

Building the Response Manually

The class is designed to be used in "streaming" scenario, building the response as it is being received:

Building the response (pseudocode)

<?php
$statusLine = read_status_line();
$response = new HTTP_Request2_Response($statusLine);

do {
    $headerLine = read_header_line();
    $response->parseHeaderLine($headerLine);
} while ($headerLine != '');

while ($chunk = read_body()) {
    $response->appendBody($chunk);
}
?>

Everything is straightforward enough, but a few things need considering:

• Constructor will throw an exception if the provided string does not look like a valid response status line.
• It is necessary to pass an empty string to parseHeaderLine() to indicate the end of response headers: this triggers some additional processing (e.g. cookie parsing).

Observers

Observers – Monitoring the request's progress

Overview

Observers are classes that can be attached to an instance of HTTP_Request2 and notified of request's progress. Possible uses:

• Drawing a progress bar for large file uploads and / or downloads;
• Saving large response body to disk instead of storing it in memory;
• Cancelling the request by throwing an exception in Observer.

HTTP_Request2 implements SplSubject interface, so Observers should implement SplObserver. When Observer is notified of an event, it should use HTTP_Request2::getLastEvent() to access event details. That method returns an associative array with 'name' and 'data' keys. Possible event names are

'connect'

On connection to remote server, 'data' is the destination (string).

'disconnect'

On disconnection from server.

'sentHeaders'

On sending the request headers, 'data' is the headers sent (string).

'sentBodyPart'

On sending a part of the request body, 'data' is the length of that part (integer).

'sentBody' (since release 2.0.0beta1)

On sending the complete request body, 'data' is the length of request body (integer).

'receivedHeaders'

On receiving the response headers, 'data' is HTTP_Request2_Response object containing these headers.

'receivedBodyPart'

On receiving a part of the response body, 'data' is the received part (string).

'receivedEncodedBodyPart'

As 'receivedBodyPart', but 'data' is still encoded by relevant Content-Encoding.

'receivedBody'

On receiving the complete response body, data is HTTP_Request2_Response object, probably containing this body.

As the events are actually sent by request Adapters, you can receive fewer or different events if you switch to another Adapter. Curl Adapter does not notify of 'connect', 'disconnect' and 'receivedEncodedBodyPart' events (it always uses 'receivedBodyPart' as cURL extension takes care of decoding). Mock Adapter does not send any notifications at all.

Usage Example

The following example shows how an Observer can be used to save response body to disk without storing it in memory. Note that only events relevant to that task are handled in its update() method, the others can be safely ignored.

The package contains another Observer implementation: HTTP_Request2_Observer_Log class that allows logging request progress to a file or an instance of Log.

Saving response body to disk

<?php
class HTTP_Request2_Observer_Download implements SplObserver
{
    protected $dir;

    protected $fp;

    public function __construct($dir)
    {
        if (!is_dir($dir)) {
            throw new Exception("'{$dir}' is not a directory");
        }
        $this->dir = $dir;
    }

    public function update(SplSubject $subject)
    {
        $event = $subject->getLastEvent();

        switch ($event['name']) {
        case 'receivedHeaders':
            if (($disposition = $event['data']->getHeader('content-disposition'))
                && 0 == strpos($disposition, 'attachment')
                && preg_match('/filename="([^"]+)"/', $disposition, $m)
            ) {
                $filename = basename($m[1]);
            } else {
                $filename = basename($subject->getUrl()->getPath());
            }
            $target = $this->dir . DIRECTORY_SEPARATOR . $filename;
            if (!($this->fp = @fopen($target, 'wb'))) {
                throw new Exception("Cannot open target file '{$target}'");
            }
            break;

        case 'receivedBodyPart':
        case 'receivedEncodedBodyPart':
            fwrite($this->fp, $event['data']);
            break;

        case 'receivedBody':
            fclose($this->fp);
        }
    }
}

$request = new HTTP_Request2(
    'http://pear.php.net/distributions/manual/pear_manual_en.tar.bz2',
    HTTP_Request2::METHOD_GET, array('store_body' => false)
);
$request->attach(new HTTP_Request2_Observer_Download('.'));

// This won't output anything since body isn't stored in the response
echo $request->send()->getBody();
?>

Exceptions

Exceptions – Handling package errors

Exceptions overview

All exceptions thrown in HTTP_Request2 are instances of HTTP_Request2_Exception. Since release 2.0.0beta1, HTTP_Request2 tries to throw a specialized subclass of that class and provide an error code when possible.

Checking for exception subclass and error code can help your application identify transient failures (e.g. HTTP_Request2_MessageException with error code HTTP_Request2_Exception::TIMEOUT). You can also use this information to display a user friendly error message instead of displaying Exception message that is more programmer friendly.

The following HTTP_Request2_Exception subclasses are available:

HTTP_Request2_NotImplementedException

Exception thrown in case of missing package features.

HTTP_Request2_LogicException

Exception that represents error in the program logic. These are usually thrown before request even starts. This exception implies an error on part of the programmer, like passing an invalid argument to a method or trying to use Curl Adapter when curl extension is disabled. They are unlikely to happen in production except for those dealing with local files.

HTTP_Request2_ConnectionException

Exception thrown when connection to a web or proxy server fails.

HTTP_Request2_MessageException

Thrown when sending or receiving HTTP message fails (this implies that connection succeeded, at least). Can be caused by network problems (e.g. timeout) or remote server sending invalid data.

Error codes

Subclasses of HTTP_Request2_Exception can contain two error codes:

• Package error code, one of the constants described below. Can be accessed via getCode() method.
• Native error code, as returned by an underlying PHP extension used by the Adapter. Specifically, these are the error codes returned by stream_socket_client() for Socket Adapter and curl_errno() for Curl Adapter. Native error code can be accessed via HTTP_Request2_Exception::getNativeCode().

The following package error codes are currently used: