| Updated: January 4, 2001 (http://curl.haxx.se/docs/faq.shtml) |
| _ _ ____ _ |
| ___| | | | _ \| | |
| / __| | | | |_) | | |
| | (__| |_| | _ <| |___ |
| \___|\___/|_| \_\_____| |
| |
| FAQ |
| |
| 1. Philosophy |
| 1.1 What is cURL? |
| 1.2 What is libcurl? |
| 1.3 What is cURL not? |
| 1.4 When will you make curl do XXXX ? |
| |
| 2. Install Related Problems |
| 2.1 configure doesn't find OpenSSL even when it is installed |
| 2.1.1. native linker doesn't find openssl |
| 2.1.2. only the libssl lib is missing |
| 2.2 Does curl work/build with other SSL libraries? |
| 2.3 Where can I find a copy of LIBEAY32.DLL? |
| 2.4 Does cURL support Socks (RFC 1928) ? |
| |
| 3. Usage Problems |
| 3.1 curl: (1) SSL is disabled, https: not supported |
| 3.2 How do I tell curl to resume a transfer? |
| 3.3 Why doesn't my posting using -F work? |
| 3.4 How do I tell curl to run custom FTP commands? |
| 3.5 How can I disable the Pragma: nocache header? |
| 3.6 Does curl support javascript, ASP, XML, XHTML or HTML version Y? |
| 3.7 Can I use curl to delete/rename a file through FTP? |
| 3.8 How do I tell curl to follow HTTP redirects? |
| |
| 4. Running Problems |
| 4.1 Problems connecting to SSL servers. |
| 4.2 Why do I get problems when I use & in the URL? |
| 4.3 How can I use {, }, [ or ] to specify multiple URLs? |
| 4.4 Why do I get downloaded data even though the web page doesn't exist? |
| 4.5 Why do I get return code XXX from a HTTP server? |
| 4.5.1 "400 Bad Request" |
| 4.5.2 "401 Unauthorized" |
| 4.5.3 "403 Forbidden" |
| 4.5.4 "404 Not Found" |
| 4.5.5 "405 Method Not Allowed" |
| 4.6 Can you tell me what error code 142 means? |
| 4.7 How do I keep usernames and passwords secret in Curl command lines? |
| 4.8 I found a bug! |
| 4.9 Curl can't authenticate to the server that requires NTLM? |
| |
| 5. libcurl Issues |
| 5.1 Is libcurl thread safe? |
| 5.2 How can I receive all data into a large memory chunk? |
| 5.3 How do I fetch multiple files with libcurl? |
| 5.4 Does libcurl do Winsock initing on win32 systems? |
| |
| 6. License Issues |
| 6.1 I have a GPL program, can I use the libcurl library? |
| 6.2 I have a closed-source program, can I use the libcurl library? |
| 6.3 I have a BSD licensed program, can I use the libcurl library? |
| 6.4 I have a program that uses LGPL libraries, can I use libcurl? |
| 6.5 Can I modify curl/libcurl for my program and keep the changes secret? |
| 6.6 Can you please change the curl/libcurl license to XXXX? |
| |
| ============================================================================== |
| |
| 1. Philosophy |
| |
| 1.1 What is cURL? |
| |
| cURL (or simply just 'curl') is a command line tool for getting or sending |
| files using URL syntax. The name is a play on 'Client for URLs', originally |
| with URL spelled in uppercase to make it obvious it deals with URLs. The |
| fact it can also be pronounced 'see URL' also helped. |
| |
| Curl supports a range of common internet protocols, currently including |
| HTTP, HTTPS, FTP, GOPHER, LDAP, DICT and FILE. |
| |
| Please spell it cURL or just curl. |
| |
| 1.2 What is libcurl? |
| |
| libcurl is the engine inside curl that does all the work. curl is more or |
| less the command line interface that converts the given options into libcurl |
| function invokes. libcurl is a reliable, higly portable multiprotocol file |
| transfer library. |
| |
| Any application is free to use libcurl, even commercial or closed-source |
| ones. Just make sure changes to the lib itself are made public. |
| |
| 1.3 What is cURL not? |
| |
| Curl is *not*, I repeat, *not* a wget clone even though that is a very |
| common misconception. Never, during curl's development, have I intended curl |
| to replace wget or compete on its market. Curl is targeted at single-shot |
| file transfers. |
| |
| Curl is not a web site mirroring program. If you wanna use curl to mirror |
| something: fine, go ahead and write a script that wraps around curl to make |
| it reality (like curlmirror.pl does). |
| |
| Curl is not an ftp site mirroring program. Sure, get and send ftp with curl |
| but if you want systematic and sequential behaviour you should write a |
| script (or write a new program that interfaces libcurl) and do it. |
| |
| Curl is not a PHP tool, even though it works perfectly well when used from |
| or with PHP. |
| |
| Curl is not a single-OS program. Curl exists, compiles, builds and runs |
| under a wide range of operating systems, including all modern Unixes, |
| Windows, Amiga, BeOS, OS/2, OS X, QNX etc. |
| |
| 1.4 When will you make curl do XXXX ? |
| |
| I love suggestions of what to change in order to make curl and libcurl |
| better. I do however believe in a few rules when it comes to the future of |
| curl: |
| |
| * It is to remain a command line tool. If you want GUIs or fancy scripting |
| capabilities, you're free to write another tool that uses libcurl and that |
| offers this. There's no point in having one single tool that does every |
| imaginable thing. That's also one of the great advantages of having the |
| core of curl as a library: libcurl. |
| |
| * I do not add things to curl that other small and available tools already |
| do very fine at the side. Curl's output is fine to pipe into another |
| program or redirect to another file for the next program to interpret. |
| |
| * I focus on protocol related issues and improvements. If you wanna do more |
| magic with the supported protocols than curl currently does, chances are |
| big I will agree. If you wanna add more protocols, I may very well |
| agree. |
| |
| * If you want me to make all the work while you wait for me to implement it |
| for you, that is not a very friendly attitude. I spend a considerable time |
| already on maintaining and developing curl. In order to get more out of |
| me, I trust you will offer some of your time and efforts in return. |
| |
| * If you write the code, chances are bigger that it will get into curl |
| faster. |
| |
| 2. Install Related Problems |
| |
| 2.1. configure doesn't find OpenSSL even when it is installed |
| |
| This may be because of several reasons. |
| |
| 2.1.1. native linker doesn't find openssl |
| |
| Affected platforms: |
| Solaris (native cc compiler) |
| HPUX (native cc compiler) |
| SGI IRIX (native cc compiler) |
| SCO UNIX (native cc compiler) |
| |
| When configuring curl, I specify --with-ssl. OpenSSL is installed in |
| /usr/local/ssl Configure reports SSL in /usr/local/ssl, but fails to find |
| CRYPTO_lock in -lcrypto |
| |
| Cause: The cc for this test places the -L/usr/local/ssl/lib AFTER |
| -lcrypto, so ld can't find the library. This is due to a bug in the GNU |
| autoconf tool. |
| |
| Workaround: Specifying "LDFLAGS=-L/usr/local/ssl/lib" in front of |
| ./configure places the -L/usr/local/ssl/lib early enough in the command |
| line to make things work |
| |
| Solution submitted by: Bob Allison <allisonb@users.sourceforge.net> |
| |
| 2.1.2. only the libssl lib is missing |
| |
| If all include files and the libcrypto lib is present, with only the |
| libssl being missing according to configure, this is mostly likely because |
| a few functions are left out from the libssl. |
| |
| If the function names missing include RSA or RSAREF you can be certain |
| that this is because libssl requires the RSA and RSASEF libs to build. |
| |
| See the INSTALL file section that explains how to add those libs to |
| configure. Make sure that you remove the config.cache file before you |
| rerun configure with the new flags. |
| |
| 2.2. Does curl work/build with other SSL libraries? |
| |
| Curl has been written to use OpenSSL, although I doubt there would be much |
| problems using a different library. I just don't know any other free one and |
| that has limited my possibilities to develop against anything else. |
| |
| If anyone does "port" curl to use a commercial SSL library, I am of course |
| very interested in getting the patch! |
| |
| 2.3. Where can I find a copy of LIBEAY32.DLL? |
| |
| That is an OpenSSL binary built for Windows. |
| |
| Curl uses OpenSSL to do the SSL stuff. The LIBEAY32.DLL is what curl needs |
| on a windows machine to do https://. Check out the curl web page to find |
| accurate and up-to-date pointers to recent OpenSSL DDLs and other binary |
| packages. |
| |
| 2.4. Does cURL support Socks (RFC 1928) ? |
| |
| No. Nobody has wanted it that badly yet. I would appriciate patches that |
| brings this functionality. |
| |
| |
| 3. Usage problems |
| |
| 3.1. curl: (1) SSL is disabled, https: not supported |
| |
| If you get this output when trying to get anything from a https:// server, |
| it means that the configure script couldn't find all libs and include files |
| it requires for SSL to work. If the configure script fails to find them, |
| curl is simply built without SSL support. |
| |
| To get the https:// support into a curl that was previously built but that |
| reports that https:// is not supported, you should dig through the document |
| and logs and check out why the configure script doesn't find the SSL libs |
| and/or include files. |
| |
| Also, check out the other paragraph in this FAQ labeled "configure doesn't |
| find OpenSSL even when it is installed". |
| |
| 3.2. How do I tell curl to resume a transfer? |
| |
| Curl supports resume both ways on FTP, download ways on HTTP. |
| |
| Try the -C option. |
| |
| 3.3. Why doesn't my posting using -F work? |
| |
| You can't simply use -F or -d at your choice. The web server that will |
| receive your post assumes one of the formats. If the form you're trying to |
| "fake" sets the type to 'multipart/form-data', than and only then you must |
| use the -F type. In all the most common cases, you should use -d which then |
| causes a posting with the type 'application/x-www-form-urlencoded'. |
| |
| I have described this in some detail in the README.curl file, and if you |
| don't understand it the first time, read it again before you post questions |
| about this to the mailing list. I would also suggest that you read through |
| the mailing list archives for old postings and questions regarding this. |
| |
| 3.4. How do I tell curl to run custom FTP commands? |
| |
| You can tell curl to perform optional commands both before and/or after a |
| file transfer. Study the -Q/--quote option. |
| |
| Since curl is used for file transfers, you don't use curl to just perform |
| ftp commands without transfering anything. Therefore you must always specify |
| a URL to transfer to/from even when doing custom FTP commands. |
| |
| 3.5. How can I disable the Pragma: nocache header? |
| |
| You can change all internally generated headers by adding a replacement with |
| the -H/--header option. By adding a header with empty contents you safely |
| disable that one. Use -H "Pragma:" to disable that specific header. |
| |
| 3.6. Does curl support javascript, ASP, XML, XHTML or HTML version Y? |
| |
| To curl, all contents are alike. It doesn't matter how the page was |
| generated. It may be ASP, PHP, perl, shell-script, SSI or plain |
| HTML-files. There's no difference to curl and it doesn't even know what kind |
| of language that generated the page. |
| |
| Javascript is slightly different since that is code embedded in the HTML |
| that is sent for the client to interpret and curl has no javascript |
| interpreter. |
| |
| 3.7. Can I use curl to delete/rename a file through FTP? |
| |
| Yes. You specify custom ftp commands with -Q/--quote. |
| |
| One example would be to delete a file after you have downloaded it: |
| |
| curl -O ftp://download.com/coolfile -Q '-DELE coolfile' |
| |
| 3.8 How do I tell curl to follow HTTP redirects? |
| |
| Curl does not follow so-called redirects by default. The Location: header |
| that informs the client about this is only interpreted if you're using the |
| -L/--location option. As in: |
| |
| curl -L http://redirector.com |
| |
| 4. Running Problems |
| |
| 4.1. Problems connecting to SSL servers. |
| |
| It took a very long time before I could sort out why curl had problems |
| to connect to certain SSL servers when using SSLeay or OpenSSL v0.9+. |
| The error sometimes showed up similar to: |
| |
| 16570:error:1407D071:SSL routines:SSL2_READ:bad mac decode:s2_pkt.c:233: |
| |
| It turned out to be because many older SSL servers don't deal with SSLv3 |
| requests properly. To correct this problem, tell curl to select SSLv2 from |
| the command line (-2/--sslv2). |
| |
| I have also seen examples where the remote server didn't like the SSLv2 |
| request and instead you had to force curl to use SSLv3 with -3/--sslv3. |
| |
| 4.2. Why do I get problems when I use & in the URL? |
| |
| In general unix shells, the & letter is treated special and when used it |
| runs the specified command in the background. To safely send the & as a part |
| of a URL, you should qoute the entire URL by using single (') or double (") |
| quotes around it. |
| |
| An example that would invoke a remote CGI that uses &-letters could be: |
| |
| curl 'http://www.altavista.com/cgi-bin/query?text=yes&q=curl' |
| |
| 4.3. How can I use {, }, [ or ] to specify multiple URLs? |
| |
| Because those letters have a special meaning to the shell, and to be used in |
| a URL specified to curl you must quote them. |
| |
| An example that downloads two URLs (sequentially) would do: |
| |
| curl '{curl,www}.haxx.se' |
| |
| 4.4. Why do I get downloaded data even though the web page doesn't exist? |
| |
| Curl asks remote servers for the page you specify. If the page doesn't exist |
| at the server, the HTTP protocol defines how the server should respond and |
| that means that headers and a "page" will be returned. That's simply how |
| HTTP works. |
| |
| By using the --fail option you can tell curl explicitly to not get any data |
| if the HTTP return code doesn't say success. |
| |
| 4.5 Why do I get return code XXX from a HTTP server? |
| |
| RFC2616 clearly explains the return codes. I'll make a short transcript |
| here. Go read the RFC for exact details: |
| |
| 4.5.1 "400 Bad Request" |
| |
| The request could not be understood by the server due to malformed |
| syntax. The client SHOULD NOT repeat the request without modifications. |
| |
| 4.5.2 "401 Unauthorized" |
| |
| The request requires user authentication. |
| |
| 4.5.3 "403 Forbidden" |
| |
| The server understood the request, but is refusing to fulfill it. |
| Authorization will not help and the request SHOULD NOT be repeated. |
| |
| 4.5.4 "404 Not Found" |
| |
| The server has not found anything matching the Request-URI. No indication |
| is given of whether the condition is temporary or permanent. |
| |
| 4.5.5 "405 Method Not Allowed" |
| |
| The method specified in the Request-Line is not allowed for the resource |
| identified by the Request-URI. The response MUST include an Allow header |
| containing a list of valid methods for the requested resource. |
| |
| 4.6. Can you tell me what error code 142 means? |
| |
| All error codes that are larger than the highest documented error code means |
| that curl has existed due to a timeout. There was no nice way for curl to |
| abort from such a condition and that's why it got this undocumented |
| error. This should not occur in releases after 7.4.1. |
| |
| 4.7. How do I keep usernames and passwords secret in Curl command lines? |
| |
| I see this problem as two parts: |
| |
| The first part is to avoid having clear-text passwords in the command line |
| so that they don't appear in 'ps' outputs and similar. That is easily |
| avoided by using the "-K" option tho tell curl to read parameters from a |
| file or stdin to which you can pass the secret info. |
| |
| To keep the passwords in your account secret from the rest of the world is |
| not a task that curl addresses. You could of course encrypt them somehow to |
| at least hide them from being read by human eyes, but that is not what |
| anyone would call security. |
| |
| Also note that regular HTTP and FTP passwords are sent in clear across the |
| network. All it takes for anyone to fetch them is to listen on the network. |
| Evesdropping is very easy. |
| |
| 4.8 I found a bug! |
| |
| It is not a bug if the behaviour is documented. Read the docs first. |
| |
| If it is a problem with a binary you've downloaded or a package for your |
| particular platform, try contacting the person who built the package/archive |
| you have. |
| |
| If there is a bug, post a bug report in the Curl Bug Track System over at |
| http://sourceforge.net/bugs/?group_id=976 or mail a detailed bug description |
| to curl-bug@haxx.se. |
| |
| Always include as many details you can think of, including curl version, |
| operating system name and version and complete instructions how to repeat |
| the bug. |
| |
| 4.9. Curl can't authenticate to the server that requires NTLM? |
| |
| NTLM is a Microsoft proprietary protocol. Unfortunately, curl does not |
| currently support that. |
| |
| 5. libcurl Issues |
| |
| 5.1. Is libcurl thread safe? |
| |
| We have attempted to write the entire code adjusted for multi-threaded |
| programs. If your system has such, curl will attempt to use threadsafe |
| functions instead of non-safe ones. |
| |
| I am very interested in once and for all getting some kind of report or |
| README file from those who have used libcurl in a threaded environment, |
| since I haven't and I get this question more and more frequently! |
| |
| 5.2 How can I receive all data into a large memory chunk? |
| |
| You are in full control of the callback function that gets called every time |
| there is data received from the remote server. You can make that callback do |
| whatever you want. You do not have to write the receivied data to a file. |
| |
| One solution to this problem could be to have a pointer to a struct that you |
| pass to the callback function. You set the pointer using the |
| curl_easy_setopt(CURLOPT_FILE) function. Then that pointer will be passed to |
| the callback instead of a FILE * to a file: |
| |
| /* imaginary struct */ |
| struct MemoryStruct { |
| char *memory; |
| size_t size; |
| }; |
| |
| /* imaginary callback function */ |
| size_t |
| WriteMemoryCallback(void *ptr, size_t size, size_t nmemb, void *data) |
| { |
| register int realsize = size * nmemb; |
| struct MemoryStruct *mem = (struct MemoryStruct *)data; |
| |
| mem->memory = (char *)realloc(mem->memory, mem->size + realsize + 1); |
| if (mem->memory) { |
| memcpy(&(mem->memory[mem->size]), ptr, realsize); |
| mem->size += realsize; |
| mem->memory[mem->size] = 0; |
| } |
| return realsize; |
| } |
| |
| 5.3 How do I fetch multiple files with libcurl? |
| |
| The easy interface of libcurl does not support multiple requests using the |
| same connection. The only available way to do multiple requests is to |
| init/perform/cleanup for each request. |
| |
| 5.4 Does libcurl do Winsock initing on win32 systems? |
| |
| No. |
| |
| On win32 systems, you need to init the winsock stuff manually, libcurl will |
| not do that for you. WSAStartup() and WSACleanup() should be used |
| accordingly. The reason for this is of course that a single application may |
| use several different libraries and parts, and there's no reason for every |
| single library to do this. |
| |
| 6. License Issues |
| |
| NOTE: This section is now updated to concern curl 7.5.2 or later! |
| |
| Curl and libcurl are released under a MIT/X derivate license *or* the MPL, |
| the Mozilla Public License. To get a really good answer to your license |
| conflict questions, you should study the MPL and MIT/X licenses and the |
| license you are about to use and check for clashes yourself. This section is |
| just a brief summary for the cases we get the most questions. (Parts of this |
| section was much enhanced by Bjorn Reese.) |
| |
| 6.1. I have a GPL program, can I use the libcurl library? |
| |
| Yes! |
| |
| Since libcurl may be distributed under the MIT/X derivate license, it can be |
| used together with GPL in any software. |
| |
| 6.2. I have a closed-source program, can I use the libcurl library? |
| |
| Yes. |
| |
| libcurl does not put any restrictions on the program that uses the |
| library. |
| |
| 6.3. I have a BSD licensed program, can I use the libcurl library? |
| |
| Yes. |
| |
| libcurl does not put any restrictions on the program that uses the |
| library. |
| |
| 6.4. I have a program that uses LGPL libraries, can I use libcurl? |
| |
| Yes. |
| |
| The LGPL license don't clash with other licenses. |
| |
| 6.5. Can I modify curl/libcurl for my program and keep the changes secret? |
| |
| Yes. |
| |
| The MIT/X derivate license practically allows you to do almost anything with |
| the sources, on the condition that the copyright texts in the sources are |
| left intact. |
| |
| 6.6. Can you please change the curl/libcurl license to XXXX? |
| |
| No. We carefully picked this license years ago and a large amount of people |
| have contributed with source code knowing that this is the license we |
| use. This license puts the restrictions we want on curl/libcurl and it does |
| not spread to other programs or libraries that use it. The recent dual |
| license modification should make it possible for everyone to use libcurl or |
| curl in their projects, no matter what license they already have in use. |