| The test suite's file format is very simple and extensible, closely |
| resembling XML. All data for a single test case resides in a single |
| ASCII file. Labels mark the beginning and the end of all sections, and each |
| label must be written in its own line. Comments are either XML-style |
| (enclosed with <!-- and -->) or C-style (beginning with #) and must appear |
| on their own lines and not alongside actual test data. Most test data files |
| are syntactically valid XML, although a few files are not (lack of |
| support for character entities and the preservation of CR/LF characters at |
| the end of lines are the biggest differences). |
| |
| The file begins with a 'testcase' tag, which encompasses the remainder of |
| the file. |
| |
| <testcase> |
| |
| Each file is split up in three main sections: reply, client and verify. The |
| reply section is used for the server to know what to send as a reply for the |
| requests curl sends, the client section defines how the client should behave |
| while the verify section defines how to verify that the data stored after a |
| command has been run ended up correctly. |
| |
| Each main section has a number of available subsections that can be |
| specified, that will be checked/used if specified. This document includes all |
| the subsections currently supported. |
| |
| Main sections are 'info', 'reply', 'client' and 'verify'. |
| |
| <info> |
| <keywords> |
| A newline-separated list of keywords describing what this test case uses and |
| tests. Try to use an already used keyword. These keywords will be used for |
| statistical/informational purposes and for choosing or skipping classes |
| of tests. "Keywords" must begin with an alphabetic character, "-", "[" |
| or "{" and may actually consist of multiple words separated by spaces |
| which are treated together as a single identifier. |
| </keywords> |
| </info> |
| |
| <reply> |
| <data [nocheck="yes"] [sendzero="yes"] [base64="yes"]> |
| data to be sent to the client on its request and later verified that it arrived |
| safely. Set nocheck="yes" to prevent the test script from verifying the arrival |
| of this data. |
| |
| If the data contains 'swsclose' anywhere within the start and end tag, and |
| this is a HTTP test, then the connection will be closed by the server after |
| this response is sent. If not, the connection will be kept persistent. |
| |
| If the data contains 'swsbounce' anywhere within the start and end tag, the |
| HTTP server will detect if this is a second request using the same test and |
| part number and will then increase the part number with one. This is useful |
| for auth tests and similar. |
| |
| 'sendzero' set to yes means that the (FTP) server will "send" the data even if |
| the size is zero bytes. Used to verify curl's behaviour on zero bytes |
| transfers. |
| |
| 'base64' set to yes means that the data provided in the test-file is a chunk |
| of data encoded with base64. It is the only way a test case can contain binary |
| data. (This attribute can in fact be used on any section, but it doesn't make |
| much sense for other sections than "data"). |
| |
| For FTP file listings, the <data> section will be used *only* if you make sure |
| that there has been a CWD done first to a directory named 'test-[num]' where |
| [num] is the test case number. Otherwise the ftp server can't know from which |
| test file to load the list content. |
| |
| </data> |
| <dataNUM> |
| Send back this contents instead of the <data> one. The num is set by: |
| A) The test number in the request line is >10000 and this is the remainder |
| of [test case number]%10000. |
| B) The request was HTTP and included digest details, which adds 1000 to NUM |
| C) If a HTTP request is NTLM type-1, it adds 1001 to num |
| D) If a HTTP request is NTLM type-3, it adds 1002 to num |
| E) If a HTTP request is Basic and num is already >=1000, it adds 1 to num |
| |
| Dynamically changing num in this way allows the test harness to be used to |
| test authentication negotiation where several different requests must be sent |
| to complete a transfer. The response to each request is found in its own data |
| section. Validating the entire negotiation sequence can be done by |
| specifying a datacheck section. |
| </dataNUM> |
| <connect> |
| The connect section is used instead of the 'data' for all CONNECT |
| requests. The remainder of the rules for the data section then apply but with |
| a connect prefix. |
| </connect> |
| <datacheck [mode="text"] [nonewline="yes"]> |
| if the data is sent but this is what should be checked afterwards. If |
| 'nonewline' is set, we will cut off the trailing newline of this given data |
| before comparing with the one actually received by the client. |
| |
| Use the mode="text" attribute if the output is in text mode on platforms that |
| have a text/binary difference. |
| </datacheck> |
| <datacheckNUM [nonewline="yes"] [mode="text"]> |
| The contents of numbered datacheck sections are appended to the non-numbered |
| one. |
| </datacheckNUM> |
| <size> |
| number to return on a ftp SIZE command (set to -1 to make this command fail) |
| </size> |
| <mdtm> |
| what to send back if the client sends a (FTP) MDTM command, set to -1 to |
| have it return that the file doesn't exist |
| </mdtm> |
| <postcmd> |
| special purpose server-command to control its behavior *after* the |
| reply is sent |
| For HTTP/HTTPS, these are supported: |
| |
| wait [secs] |
| - Pause for the given time |
| </postcmd> |
| <servercmd> |
| Special-commands for the server. |
| For FTP/SMTP/POP/IMAP, these are supported: |
| |
| REPLY [command] [return value] [response string] |
| - Changes how the server responds to the [command]. [response string] is |
| evaluated as a perl string, so it can contain embedded \r\n, for example. |
| There's a special [command] named "welcome" (without quotes) which is the |
| string sent immediately on connect as a welcome. |
| COUNT [command] [num] |
| - Do the REPLY change for [command] only [num] times and then go back to the |
| built-in approach |
| DELAY [command] [secs] |
| - Delay responding to this command for the given time |
| RETRWEIRDO |
| - Enable the "weirdo" RETR case when multiple response lines appear at once |
| when a file is transferred |
| RETRNOSIZE |
| - Make sure the RETR response doesn't contain the size of the file |
| NOSAVE |
| - Don't actually save what is received |
| SLOWDOWN |
| - Send FTP responses with 0.01 sec delay between each byte |
| PASVBADIP |
| - makes PASV send back an illegal IP in its 227 response |
| CAPA [capabilities] |
| - Enables support for and specifies a list of space separated capabilities to |
| return to the client for the IMAP CAPABILITY, POP3 CAPA and SMTP EHLO |
| commands |
| AUTH [mechanisms] |
| - Enables support for SASL authentication and specifies a list of space |
| separated mechanisms for IMAP, POP3 and SMTP |
| |
| For HTTP/HTTPS: |
| auth_required if this is set and a POST/PUT is made without auth, the |
| server will NOT wait for the full request body to get sent |
| idle do nothing after receiving the request, just "sit idle" |
| stream continuously send data to the client, never-ending |
| writedelay: [secs] delay this amount between reply packets |
| pipe: [num] tell the server to expect this many HTTP requests before |
| sending back anything, to allow pipelining tests |
| skip: [num] instructs the server to ignore reading this many bytes from a PUT |
| or POST request |
| |
| rtp: part [num] channel [num] size [num] |
| stream a fake RTP packet for the given part on a chosen channel |
| with the given payload size |
| |
| connection-monitor When used, this will log [DISCONNECT] to the server.input |
| log when the connection is disconnected. |
| upgrade when an HTTP upgrade header is found, the server will upgrade |
| to http2 |
| |
| For TFTP: |
| writedelay: [secs] delay this amount between reply packets (each packet being |
| 512 bytes payload) |
| </servercmd> |
| </reply> |
| |
| <client> |
| |
| <server> |
| What server(s) this test case requires/uses: |
| |
| file |
| ftp |
| ftp-ipv6 |
| ftps |
| http |
| http-ipv6 |
| http-pipe |
| http-proxy |
| http-unix |
| https |
| httptls+srp |
| httptls+srp-ipv6 |
| http/2 |
| imap |
| none |
| pop3 |
| rtsp |
| rtsp-ipv6 |
| scp |
| sftp |
| smtp |
| socks4 |
| socks5 |
| |
| Give only one per line. This subsection is mandatory. |
| </server> |
| |
| <features> |
| A list of features that MUST be present in the client/library for this test to |
| be able to run. If a required feature is not present then the test will be |
| SKIPPED. |
| |
| Alternatively a feature can be prefixed with an exclamation mark to indicate a |
| feature is NOT required. If the feature is present then the test will be |
| SKIPPED. |
| |
| Features testable here are: |
| |
| axTLS |
| crypto |
| debug |
| getrlimit |
| GnuTLS |
| GSS-API |
| http2 |
| idn |
| ipv6 |
| Kerberos |
| large_file |
| libz |
| Metalink |
| NSS |
| NTLM |
| OpenSSL |
| PSL |
| socks |
| SPNEGO |
| SSL |
| SSLpinning |
| SSPI |
| TLS-SRP |
| TrackMemory |
| unittest |
| unix-sockets |
| WinSSL |
| |
| as well as each protocol that curl supports. A protocol only needs to be |
| specified if it is different from the server (useful when the server |
| is 'none'). |
| </features> |
| |
| <killserver> |
| Using the same syntax as in <server> but when mentioned here these servers |
| are explicitly KILLED when this test case is completed. Only use this if there |
| is no other alternatives. Using this of course requires subsequent tests to |
| restart servers. |
| </killserver> |
| |
| <precheck> |
| A command line that if set gets run by the test script before the test. If an |
| output is displayed by the command or if the return code is non-zero, the test |
| will be skipped and the (single-line) output will be displayed as reason for |
| not running the test. Variables are substituted as in the <command> section. |
| </precheck> |
| |
| <postcheck> |
| A command line that if set gets run by the test script after the test. If |
| the command exists with a non-zero status code, the test will be considered |
| to have failed. Variables are substituted as in the <command> section. |
| </postcheck> |
| |
| <tool> |
| Name of tool to use instead of "curl". This tool must be built and exist |
| either in the libtest/ directory (if the tool starts with 'lib') or in the |
| unit/ directory (if the tool starts with 'unit'). |
| </tool> |
| |
| <name> |
| test case description |
| </name> |
| |
| <setenv> |
| variable1=contents1 |
| variable2=contents2 |
| |
| Set the given environment variables to the specified value before the actual |
| command is run. They are cleared again after the command has been run. |
| Variables are first substituted as in the <command> section. |
| </setenv> |
| |
| <command [option="no-output/no-include"] [timeout="secs"] [delay="secs"] |
| [type="perl"]> |
| command line to run, there's a bunch of %variables that get replaced |
| accordingly. |
| |
| Note that the URL that gets passed to the server actually controls what data |
| that is returned. The last slash in the URL must be followed by a number. That |
| number (N) will be used by the test-server to load test case N and return the |
| data that is defined within the <reply><data></data></reply> section. |
| |
| If there's no test number found above, the HTTP test server will use the |
| number following the last dot in the given hostname (made so that a CONNECT |
| can still pass on test number) so that "foo.bar.123" gets treated as test case |
| 123. Alternatively, if an IPv6 address is provided to CONNECT, the last |
| hexadecimal group in the address will be used as the test number! For example |
| the address "[1234::ff]" would be treated as test case 255. |
| |
| Set type="perl" to write the test case as a perl script. It implies that |
| there's no memory debugging and valgrind gets shut off for this test. |
| |
| Set option="no-output" to prevent the test script to slap on the --output |
| argument that directs the output to a file. The --output is also not added if |
| the verify/stdout section is used. |
| |
| Set option="no-include" to prevent the test script to slap on the --include |
| argument. |
| |
| Set timeout="secs" to override default server logs advisor read lock timeout. |
| This timeout is used by the test harness, once that the command has completed |
| execution, to wait for the test server to write out server side log files and |
| remove the lock that advised not to read them. The "secs" parameter is the not |
| negative integer number of seconds for the timeout. This 'timeout' attribute |
| is documented for completeness sake, but is deep test harness stuff and only |
| needed for very singular and specific test cases. Avoid using it. |
| |
| Set delay="secs" to introduce a time delay once that the command has completed |
| execution and before the <postcheck> section runs. The "secs" parameter is the |
| not negative integer number of seconds for the delay. This 'delay' attribute |
| is intended for very specific test cases, and normally not needed. |
| |
| Available substitute variables include: |
| %CLIENT6IP - IPv6 address of the client running curl |
| %CLIENTIP - IPv4 address of the client running curl |
| %CURL - Path to the curl executable |
| %FTP2PORT - Port number of the FTP server 2 |
| %FTP6PORT - IPv6 port number of the FTP server |
| %FTPPORT - Port number of the FTP server |
| %FTPSPORT - Port number of the FTPS server |
| %FTPTIME2 - Timeout in seconds that should be just sufficient to receive |
| a response from the test FTP server |
| %FTPTIME3 - Even longer than %FTPTIME2 |
| %GOPHER6PORT - IPv6 port number of the Gopher server |
| %GOPHERPORT - Port number of the Gopher server |
| %HOST6IP - IPv6 address of the host running this test |
| %HOSTIP - IPv4 address of the host running this test |
| %HTTP6PORT - IPv6 port number of the HTTP server |
| %HTTPPIPEPORT - Port number of the HTTP pipelining server |
| %HTTPUNIXPATH - Path to the Unix socket of the HTTP server |
| %HTTPPORT - Port number of the HTTP server |
| %HTTPSPORT - Port number of the HTTPS server |
| %HTTPTLS6PORT - IPv6 port number of the HTTP TLS server |
| %HTTPTLSPORT - Port number of the HTTP TLS server |
| %IMAP6PORT - IPv6 port number of the IMAP server |
| %IMAPPORT - Port number of the IMAP server |
| %POP36PORT - IPv6 port number of the POP3 server |
| %POP3PORT - Port number of the POP3 server |
| %PROXYPORT - Port number of the HTTP proxy |
| %PWD - Current directory |
| %RTSP6PORT - IPv6 port number of the RTSP server |
| %RTSPPORT - Port number of the RTSP server |
| %SMTP6PORT - IPv6 port number of the SMTP server |
| %SMTPPORT - Port number of the SMTP server |
| %SOCKSPORT - Port number of the SOCKS4/5 server |
| %SRCDIR - Full path to the source dir |
| %SSHPORT - Port number of the SCP/SFTP server |
| %TFTP6PORT - IPv6 port number of the TFTP server |
| %TFTPPORT - Port number of the TFTP server |
| %USER - Login ID of the user running the test |
| </command> |
| |
| <file name="log/filename"> |
| This creates the named file with this content before the test case is run, |
| which is useful if the test case needs a file to act on. |
| Variables are substituted on the contents of the file as in the <command> |
| section. |
| </file> |
| |
| <stdin [nonewline="yes"]> |
| Pass this given data on stdin to the tool. |
| |
| If 'nonewline' is set, we will cut off the trailing newline of this given data |
| before comparing with the one actually received by the client |
| </stdin> |
| |
| </client> |
| |
| <verify> |
| <errorcode> |
| numerical error code curl is supposed to return. Specify a list of accepted |
| error codes by separating multiple numbers with comma. See test 237 for an |
| example. |
| </errorcode> |
| <strip> |
| One regex per line that is removed from the protocol dumps before the |
| comparison is made. This is very useful to remove dependencies on dynamically |
| changing protocol data such as port numbers or user-agent strings. |
| </strip> |
| <strippart> |
| One perl op per line that operates on the protocol dump. This is pretty |
| advanced. Example: "s/^EPRT .*/EPRT stripped/" |
| </strippart> |
| |
| <protocol [nonewline="yes"]> |
| |
| the protocol dump curl should transmit, if 'nonewline' is set, we will cut off |
| the trailing newline of this given data before comparing with the one actually |
| sent by the client Variables are substituted as in the <command> section. The |
| <strip> and <strippart> rules are applied before comparisons are made. |
| |
| </protocol> |
| |
| <proxy [nonewline="yes"]> |
| |
| The protocol dump curl should transmit to a HTTP proxy (when the http-proxy |
| server is used), if 'nonewline' is set, we will cut off the trailing newline |
| of this given data before comparing with the one actually sent by the client |
| Variables are substituted as in the <command> section. The <strip> and |
| <strippart> rules are applied before comparisons are made. |
| |
| </proxy> |
| |
| <stdout [mode="text"] [nonewline="yes"]> |
| This verifies that this data was passed to stdout. Variables are |
| substituted as in the <command> section. |
| |
| Use the mode="text" attribute if the output is in text mode on platforms that |
| have a text/binary difference. |
| |
| If 'nonewline' is set, we will cut off the trailing newline of this given data |
| before comparing with the one actually received by the client |
| </stdout> |
| <file name="log/filename" [mode="text"]> |
| The file's contents must be identical to this after the test is complete. |
| Use the mode="text" attribute if the output is in text mode on platforms that |
| have a text/binary difference. |
| Variables are substituted as in the <command> section. |
| </file> |
| <stripfile> |
| One perl op per line that operates on the output file or stdout before being |
| compared with what is stored in the test file. This is pretty |
| advanced. Example: "s/^EPRT .*/EPRT stripped/" |
| </stripfile> |
| <upload> |
| the contents of the upload data curl should have sent |
| </upload> |
| <valgrind> |
| disable - disables the valgrind log check for this test |
| </valgrind> |
| </verify> |
| |
| </testcase> |