Send and receive binary files using PHP and cURL

22 Replies

I was recently working on a project where I had to send and receive binary files to and from a REST API, so I decided to document some of the code I wrote. Keep in mind that I’m extracting all of these code snippets from a custom PHP class I wrote, so if you’re working with an API, I’d encourage you to create a class of your own.

Setup a form to submit a file

Before you can send a binary file to an API, you have to get it from somewhere. In my project that file came from a form. The form could be as simple as this:

1
2
3
4

<form method="post" action="index.php" enctype="multipart/form-data">
    <input name="file" type="file" />
    <input type="submit" value="Upload" />
</form>
<form method="post" action="index.php" enctype="multipart/form-data">
	<input name="file" type="file" />
	<input type="submit" value="Upload" />
</form>

Note the enctype attribute is set to multipart/form-data. You must have this set, otherwise the file will not be submitted.

Grab the submitted file

Normally when you process a form, the contents are found inside of the global $_POST variable, however file data is located within the global $_FILES variable.

If you print out the contents of $_FILES on the page the form was submitted to, you’d see something like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

Array
(
    [file] => Array
        (
            [name] => Array
                (
                    [0] => 400.png
                )
            [type] => Array
                (
                    [0] => image/png
                )
            [tmp_name] => Array
                (
                    [0] => /tmp/php5Wx0aJ
                )
            [error] => Array
                (
                    [0] => 0
                )
            [size] => Array
                (
                    [0] => 15726
                )
        )
)
Array
(
    [file] => Array
        (
            [name] => Array
                (
                    [0] => 400.png
                )
            [type] => Array
                (
                    [0] => image/png
                )
            [tmp_name] => Array
                (
                    [0] => /tmp/php5Wx0aJ
                )
            [error] => Array
                (
                    [0] => 0
                )
            [size] => Array
                (
                    [0] => 15726
                )
        )
)

What’s unusual about this array is that instead of grouping each file’s data in one array, each piece of information about a file is grouped together. So the values at index 0 in name, type, tmp_name, error and size belong to one file.

The binary file data is located in a temporary storage on your server. In this case, at /tmp/php5Wx0aJ.

Send the binary contents via cURL

The following snippet sends the binary file to a URL:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

$url = 'http://some-server.com/api';
$header = array('Content-Type: multipart/form-data');
$fields = array('file' => '@' . $_FILES['file']['tmp_name'][0]);
$token = 'NfxoS9oGjA6MiArPtwg4aR3Cp4ygAbNA2uv6Gg4m';
 
$resource = curl_init();
curl_setopt($resource, CURLOPT_URL, $url);
curl_setopt($resource, CURLOPT_HTTPHEADER, $header);
curl_setopt($resource, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($resource, CURLOPT_POST, 1);
curl_setopt($resource, CURLOPT_POSTFIELDS, $fields);
curl_setopt($resource, CURLOPT_COOKIE, 'apiToken=' . $token);
$result = json_decode(curl_exec($resource));
curl_close($resource);
$url = 'http://some-server.com/api';
$header = array('Content-Type: multipart/form-data');
$fields = array('file' => '@' . $_FILES['file']['tmp_name'][0]);
$token = 'NfxoS9oGjA6MiArPtwg4aR3Cp4ygAbNA2uv6Gg4m';

$resource = curl_init();
curl_setopt($resource, CURLOPT_URL, $url);
curl_setopt($resource, CURLOPT_HTTPHEADER, $header);
curl_setopt($resource, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($resource, CURLOPT_POST, 1);
curl_setopt($resource, CURLOPT_POSTFIELDS, $fields);
curl_setopt($resource, CURLOPT_COOKIE, 'apiToken=' . $token);
$result = json_decode(curl_exec($resource));
curl_close($resource);

Let’s talk about each of these lines, because some of this will depend on the requirements of the API you are using and may not be necessary.

  • $url is where cURL will post your data.
  • $header tells the API that there is a file coming its way. You can supply more headers by adding additional values to the array.
  • $fields contains an associative array. The field name depends on what the API expects it to be. The @ sign in the value tells cURL that we’re dealing with a file.
  • $token is a token that the REST API expects as a cookie. Usually you login to the API once during a transaction, then use a token for all subsequent requests. The cookie is called apiToken, but this may differ depending on the API’s expectations.
  • CURLOPT_RETURNTRANSFER tells cURL we want a response.
  • CURLOPT_POST tells cURL we want our data to be posted.
  • CURLOPT_COOKIE sets the token as a cookie.

Lastly, notice the last line:

1

$result = json_decode(curl_exec($resource));
$result = json_decode(curl_exec($resource));

In my case I know that the response I’m getting from the API will be in the JSON format, which is why I’m decoding it. You may not need this if you’re getting back a response in XML, for example.

This completes sending a binary file via cURL. Next we’ll look at retrieving that file.

Retrieve the binary contents via cURL

Here’s how to retrieve a binary file:

1
2
3
4
5
6
7
8

$resource = curl_init();
curl_setopt($resource, CURLOPT_URL, $url);
curl_setopt($resource, CURLOPT_HEADER, 1);
curl_setopt($resource, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($resource, CURLOPT_BINARYTRANSFER, 1);
curl_setopt($resource, CURLOPT_COOKIE, 'apiToken=' . $token);
$file = curl_exec($resource);
curl_close($resource);
$resource = curl_init();
curl_setopt($resource, CURLOPT_URL, $url);
curl_setopt($resource, CURLOPT_HEADER, 1);
curl_setopt($resource, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($resource, CURLOPT_BINARYTRANSFER, 1);
curl_setopt($resource, CURLOPT_COOKIE, 'apiToken=' . $token);
$file = curl_exec($resource);
curl_close($resource);

We don’t have to go over anything except for:

  • CURLOPT_HEADER tells cURL that we expect there to be a header. This is important because it tells us what kind of file we’re getting i.e. an image, a Word document, a PDF, etc.
  • CURLOPT_BINARYTRANSFER tells PHP that the result will contain binary data. Lots of people claim you don’t need this line. I say try it with and without, just to be sure.

Now you have the binary file with its headers stored in $file.

Give the file back to the user

Let’s go over on how to give the file back to the user. You could either store the file on the server and give the user a path to the file, or you can just let PHP handle all that on the fly and simply prompt the user to save the file.

Here is how I did that:

1
2
3
4
5
6
7
8
9
10
11

$file_array = explode("\n\r", $file, 2);
$header_array = explode("\n", $file_array[0]);
foreach($header_array as $header_value) {
    $header_pieces = explode(':', $header_value);
    if(count($header_pieces) == 2) {
        $headers[$header_pieces[0]] = trim($header_pieces[1]);
    }
}
header('Content-type: ' . $headers['Content-Type']);
header('Content-Disposition: ' . $headers['Content-Disposition']);
echo substr($file_array[1], 1);
$file_array = explode("\n\r", $file, 2);
$header_array = explode("\n", $file_array[0]);
foreach($header_array as $header_value) {
	$header_pieces = explode(':', $header_value);
	if(count($header_pieces) == 2) {
		$headers[$header_pieces[0]] = trim($header_pieces[1]);
	}
}
header('Content-type: ' . $headers['Content-Type']);
header('Content-Disposition: ' . $headers['Content-Disposition']);
echo substr($file_array[1], 1);
  • On line 1 I separate the header from the rest of the file.
  • On line 2-8 I parse the header contents and put them in an array so I can easily reference the elements I need.
  • On line 9 &10 I tell PHP what the content type is and what the file should be called.
  • On line 11 I output all of the binary contents. Together with the header, it prompts the user to save the file (or displays it in the browser if it’s an image, for example).

I want to touch on the function on line 11: substr($file_array[1], 1);

In my case the binary file had an extra space at the beginning of the file. The substr function removes that. I’m not sure whether that space came from the API or from the explode on line 1, so if this doesn’t work for you, try changing that line to: echo $file_array[1];

Hopefully this info will shed some light on how to post and retrieve files in a binary format. If you have any questions, leave them in the comments below.

22 thoughts on “Send and receive binary files using PHP and cURL”

  1. Mahesh

    Hi ,

    I want to fetch a image using curl api and want to show directly in the fancy box, but it is showing in binary format always.Even I have added header there.

    The same code is working if I don’t use the fancybox.

    Can you please help me.

    Reply ↓
  2. Dev

    Hi,
    Can you suggest me on how can I verify that my file is uploaded correctly to the server?
    1) Can I send some kind of file checksum to the server using CURL? so that the server also calculates the checksum on received file and compares two checksums and reply with proper response code.

    How to do it in CURL?

    Reply ↓
    1. Ryan Post author

      You can create a script on the server that expects two values: a checksum and a filename. Then you can post those two values using a cURL. The server will then calculate the checksum of the existing file and compare it to what you sent. If it matches, you can return something like true and if it doesn’t, you can return false (and optionally remove the file). So, the first request uploads the file, and the second request verifies it.

      Reply ↓
  3. Bharat Masimukku

    Hello Ryan

    I have the following requirement
    Upload a jpg format image to jpegmini server and receive the compressed image as the response

    Please refer to the below jpegmini documentation

    Method: POST
    URL: http://ec2-54-209-163-171.compute-1.amazonaws.com/api/v1/optimize
    Body: The data of the source jpeg photo
    Command:
    curl -X POST -H ‘Content-Type: image/jpeg’ –data-binary @/tmp/input.jpg –output /tmp/output.jpg http://ec2-54-209-163-171.compute-1.amazonaws.com/api/v1/optimize

    If we break down the above command
    -X POST – specify http POST method
    -H ‘Content-Type: image/jpeg’ – specify body content is a jpeg
    –data-binary @/tmp/input.jpg – send /tmp/input.jpg as the data of the request
    –output /tmp/output.jpg – store response in /tmp/output.jpg (instead of stdout)

    Please note that the jpegmini URL is an AWS instance
    I am stopping the instance when not using it
    If you want me to run the instance please let me know

    I want to make this cURL request and receive response using PHP
    Please help me out

    Reply ↓
    1. Ryan Sechrest Post author

      Have you tried your cURL command via command line and did it give you the response you’re looking for? If so, have you already tried converting it to PHP and were unsuccessful? If so, post the PHP code you tried in a Gist so I can take a look at it.

      Reply ↓
  4. Manish

    Hello,

    As you define in your blog, I have same folllow the above steps to upload and download files using curl, but its not working anymore. Can you define by the examples. So I can understand easily.

    Thanks in Advance
    Manish Arora

    Reply ↓
    1. Ryan Sechrest Post author

      I’ll need a bit more information. What specifically isn’t working and what kind of response are you getting? When you say it isn’t working anymore, was it working for you before?

      Reply ↓
      1. Manish

        Hello Ryan,

        I use your code and as you say that download popup will display on client browser, but its not showing and display only the content of file.

        So how to download file, Please describe it here.

        Thanks in advance
        Manish

        Reply ↓
        1. Ryan Sechrest Post author

          First, I would echo out Content-Type and Content-Disposition to ensure those are right. Second, have you tried echoing out $file_array[1] without using substr()? Third, for testing, save the file to the disk, download it, and then open it to verify the file was transmitted back to you properly.

          Reply ↓
          1. phpdeveloper

            Ryan,

            Thanks for the informative post. I have been banging my head over this problem of uploading a form with file using curl but simply can’t figure out what is wrong with my code.

            Here is my code:

            1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

            $cfile = new CURLFile('/home/xyz/myimge.jpg', 'image/jpeg', 'mymage.jpg');
      
$params = array(
  "userid_id" => "some id #",
  "file" => $cfile,
  "name" => "test_file",
  "gallery" => "myalbum"
);
     
$uri = 'http://myserver.com/somedir/';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $uri);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true ); 
curl_setopt($ch, CURLOPT_POST, true );
curl_setopt($ch, CURLOPT_POSTFIELDS, $params);
       
$response_data = curl_exec($ch);
     
var_dump(curl_getinfo($ch));
        
var_dump($response_data);
curl_close( $ch );
unset( $ch );
            $cfile = new CURLFile('/home/xyz/myimge.jpg', 'image/jpeg', 'mymage.jpg');
      
$params = array(
  "userid_id" => "some id #",
  "file" => $cfile,
  "name" => "test_file",
  "gallery" => "myalbum"
);
     
$uri = 'http://myserver.com/somedir/';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $uri);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true ); 
curl_setopt($ch, CURLOPT_POST, true );
curl_setopt($ch, CURLOPT_POSTFIELDS, $params);
       
$response_data = curl_exec($ch);
     
var_dump(curl_getinfo($ch));
        
var_dump($response_data);
curl_close( $ch );
unset( $ch );

            But when I see my output from curl_getinfo, i see that content type is still “text/html”, where as all the docs I have seen suggest that, once you add a file to postfields, curl automatically converts the content type to “multipart/form-data”. I get the other form data but do not get file data where i am posting my data.

            Not sure what am I doing wrong here. Any ideas?

            I am on php 5.5, so appending ‘@’ to the file path is not working, since it is deprecated.

  5. Ryan Sechrest Post author

    @phpdeveloper

    I have not used the CURLFile class, but your syntax looks right.

    Try manually setting the header to multipart/form-data using the CURLOPT_HTTPHEADER option:

    1
2

    $header = array('Content-Type: multipart/form-data');
curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
    $header = array('Content-Type: multipart/form-data');
curl_setopt($ch, CURLOPT_HTTPHEADER, $header);

    Also, with regard to you not getting the file, you’ve checked $_FILES and it was empty?

    Reply ↓
    1. phpdeveloper

      Ryan,

      I tried manually setting the content_type to “multipart/form-data” but it has no effect.

      Like this:

      1

      curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: multipart/form-data'));
      curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: multipart/form-data'));

      Yes, $_FILES is empty.

      Reply ↓
      1. Ryan Sechrest Post author

        OK, so I setup a local environment, running PHP 5.5.12 and cURL 7.19.7, setup the following test:

        1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

        $cfile = new CURLFile('/var/www/html/image.jpg', 'image/jpeg', 'image.jpg');
 
$params = array(
  "userid_id" => "123",
  "file" => $cfile,
  "name" => "test_file",
  "gallery" => "myalbum"
);
 
$uri = 'http://server02.local.vps/upload.php';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $uri);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true );
curl_setopt($ch, CURLOPT_POST, true );
curl_setopt($ch, CURLOPT_POSTFIELDS, $params);
 
$response_data = curl_exec($ch);
 
echo print_r(curl_getinfo($ch), true);
 
echo $response_data;
 
curl_close( $ch );
unset( $ch );
        $cfile = new CURLFile('/var/www/html/image.jpg', 'image/jpeg', 'image.jpg');

$params = array(
  "userid_id" => "123",
  "file" => $cfile,
  "name" => "test_file",
  "gallery" => "myalbum"
);

$uri = 'http://server02.local.vps/upload.php';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $uri);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true );
curl_setopt($ch, CURLOPT_POST, true );
curl_setopt($ch, CURLOPT_POSTFIELDS, $params);

$response_data = curl_exec($ch);

echo print_r(curl_getinfo($ch), true);

echo $response_data;

curl_close( $ch );
unset( $ch );

        Where upload.php contained the following:

        1
2

        echo print_r($_POST, true);
echo print_r($_FILES, true);
        echo print_r($_POST, true);
echo print_r($_FILES, true);

        And I got the following output:

        Array
(
    [url] => http://server02.local.vps/upload.php
    [content_type] => text/html
    [http_code] => 200
    [header_size] => 227
    [request_size] => 199
    [filetime] => -1
    [ssl_verify_result] => 0
    [redirect_count] => 0
    [total_time] => 0.04464
    [namelookup_time] => 0.0001
    [connect_time] => 0.000162
    [pretransfer_time] => 0.000163
    [size_upload] => 52700
    [size_download] => 314
    [speed_download] => 7034
    [speed_upload] => 1180555
    [download_content_length] => 314
    [upload_content_length] => 52700
    [starttransfer_time] => 0.001441
    [redirect_time] => 0
    [redirect_url] => 
    [primary_ip] => 127.0.0.1
    [certinfo] => Array
        (
        )

)

Array
(
    [userid_id] => 123
    [name] => test_file
    [gallery] => myalbum
)

Array
(
    [file] => Array
        (
            [name] => image.jpg
            [type] => image/jpeg
            [tmp_name] => /tmp/phpYLjbUA
            [error] => 0
            [size] => 52207
        )

)

        Give the code a try and see if you get a different result. If this doesn’t work, make sure image permissions are correct, try a different (perhaps smaller) image, and make sure PHP can write to the tmp directory. I’m basically thinking it’s something other than your code.

        PS: I noticed myimge.jpg was misspelled, so make sure that there is a file by that name (missing the letter ‘a’ in image).

        Reply ↓
        1. phpdeveloper

          Ryan,

          Thanks for your help.

          Your code looks same as mine and yetl I get different result. I am wondering if it has to do with the framework I am using. I am using Kohana 3.3 and capturing the request object in my api to get the post variables. I am going to see if there is some issue on that front. Please let me know if you have any experience with Kohana.

          Reply ↓
  6. phpdeveloper

    Ryan,

    I started from the beginning, deleted all my code. Then copied your code and tested it, and it worked. 🙂 Looks like I had something in the code which was making it not work right. Not sure what it was though.
    Thanks a lot for your help.

    Reply ↓
  7. Brian Anderson

    Ryan,
    I hope you’re still monitoring this thread.
    Thank you in advance for reviewing this!
    I’m feeling pretty stupid about this and am willing to pay for somebody else’s smarts on a consulting deal.

    I’m trying to write a receiver for the following curl command line:
    curl -d @it20010002.xml -H “Content-Type: application/xml” https://upofx.proautomation.com/w2uploadtest.php > it20010002e.xml

    Here is my sender code
    $cfile,
    “name” => “it20210001.xml”
    );

    $resource = curl_init();
    curl_setopt($resource, CURLOPT_URL, $url);
    // curl_setopt($resource, CURLOPT_HTTPHEADER, $header);
    curl_setopt($resource, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($resource, CURLOPT_POST, 1);
    curl_setopt($resource, CURLOPT_POSTFIELDS, $params);
    $result = curl_exec($resource);
    var_dump(curl_getinfo($resource));
    echo “result=”;
    var_dump($result);
    curl_close($resource);
    echo “——result=$result—–“;
    ?>

    Here is my receiver code:
    <?php

    echo "inside w2uploadtest”;

    $uploaddir = realpath(‘./’) . ‘/’;
    $uploadfile = $uploaddir . basename($_FILES[‘file’][‘name’]);
    echo ‘Here is what happens’;
    if (move_uploaded_file($_FILES[‘file’][‘tmp_name’], $uploadfile)) {
    echo “File is valid, and was successfully uploaded.\n”;
    } else {
    echo “Possible file upload attack!\n”;
    }
    echo ‘Here is some more debugging info:’;
    print_r($_FILES);
    echo “\n\n”;
    print_r($_POST);
    echo “returning back”;
    ?>

    my variables returned are:
    array (size=26)
    ‘url’ => string ‘https://upofx.proautomation.com/w2uploadtest.php’ (length=48)
    ‘content_type’ => null
    ‘http_code’ => int 0
    ‘header_size’ => int 0
    ‘request_size’ => int 0
    ‘filetime’ => int -1
    ‘ssl_verify_result’ => int 0
    ‘redirect_count’ => int 0
    ‘total_time’ => float 0.109
    ‘namelookup_time’ => float 0
    ‘connect_time’ => float 0.031
    ‘pretransfer_time’ => float 0
    ‘size_upload’ => float 0
    ‘size_download’ => float 0
    ‘speed_download’ => float 0
    ‘speed_upload’ => float 0
    ‘download_content_length’ => float -1
    ‘upload_content_length’ => float -1
    ‘starttransfer_time’ => float 0
    ‘redirect_time’ => float 0
    ‘redirect_url’ => string ” (length=0)
    ‘primary_ip’ => string ‘50.246.192.200’ (length=14)
    ‘certinfo’ =>
    array (size=0)
    empty
    ‘primary_port’ => int 443
    ‘local_ip’ => string ‘192.168.0.44’ (length=12)
    ‘local_port’ => int 59452

    result=boolean false

    ——
    result=
    —–
    postResult=

    Reply ↓
  8. Ikram

    Hi Ryan,

    I am trying to use the PUT method to send 2 files to a REST API. I am using following code:

    1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

    $sURL = "http://some_valid_url";
$header = array('Content-Type: multipart/form-data');
$fields = array(
                'file1' => '@' . $xmlFilePath,
                'file2' => '@' . $fileFilePath
        );
 
$resource = curl_init();
curl_setopt($resource, CURLOPT_URL, $sURL);
curl_setopt($resource, CURLOPT_HTTPHEADER, $header);
curl_setopt($resource, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($resource, CURLOPT_PUT, 1);
curl_setopt($resource, CURLOPT_POSTFIELDS, $fields);
 
$result = curl_exec($resource);
curl_close($resource);
print_r($result); //print response
    $sURL = "http://some_valid_url";
$header = array('Content-Type: multipart/form-data');
$fields = array(
                'file1' => '@' . $xmlFilePath,
                'file2' => '@' . $fileFilePath
        );

$resource = curl_init();
curl_setopt($resource, CURLOPT_URL, $sURL);
curl_setopt($resource, CURLOPT_HTTPHEADER, $header);
curl_setopt($resource, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($resource, CURLOPT_PUT, 1);
curl_setopt($resource, CURLOPT_POSTFIELDS, $fields);

$result = curl_exec($resource);
curl_close($resource);
print_r($result); //print response

    The server is generating error.

    Error:
{"Message":"Expected MIME multipart request."}

    Please, guide

    Reply ↓
  9. Jignesh Boricha

    Hi Ryan,

    Thank you for your brief explanation. It has saved my day. My team mates were having so many issues but your post which I have found saved them a lot. We are very thankful to you.

    Regards.
    Thank you

    Reply ↓

Leave a Reply

Your email address will not be published. Required fields are marked *