CLI utility that does several web protocols, including HTTP, FTP, SMTP, DICT.
More powerful than wget: only use wget for recursive mirroring.
Not POSIX, and there is no POSIX 7 alternative: http://stackoverflow.com/questions/9490872/is-wget-or-similar-programs-always-available-on-posix-systems.
Ubuntu install:
sudo aptitude install -y curl
Make a GET / HTTP/1.1 request to Google, wait for response, and print response:
curl google.com
cURL does not have a dry-run option built-in: http://stackoverflow.com/questions/6180162/echo-curl-request-header-body-without-sending-it/6180363#6180363
There are however a few options to visualize what it is doing:
-
-vand other verbosity options. -
using
nc -land curl-m 1:nc -l localhost 8000 & curl -m 1 localhost:8000
-v , --trace "$FILE", --trace-ascii "$FILE": increasing levels of log verbosity.
- to stdout.
Print all data IO and curl status:
curl --trace - "$URL"
Good way to see what is going on.
curl -Lv google.com
Timeout for entire operation.
Follow redirects.
Omit redirect page if any.
Example:
curl google.pn
curl -L google.pn
Good example if you are not one of the 100 people who live in Pitcairn island =): Google redirects you to your countries domain.
With -v you can see the full transaction:
curl -vL google.pn
Make POST request:
curl -d "a=1" "$URL"
Data from stdin with -d @-:
echo 'a=1' | curl -d @- "$URL"
Multiple data are joined by an ampersand &:
curl -d 'a=1' -d 'b=2' "$URL"
Multipart POST request like done from an HTML form by a browser:
echo "Content of a.txt" > a.txt
curl -F "key1=val1" -F "file1=@a.txt" "$URL"
Custom header.
Overrides default cURL headers.
curl -d '{"a":"b"}' -H "Content-Type:application/json" "$URL"
Show received HTTP headers.
Example:
curl -i google.com
TODO vs -D -
Make HTTP HEAD request:
curl -I google.com
Implies -i of course.
Use custom HTTP method:
curl -X 'GET' google.com
Many methods have an specific option for them.
Encode spaces and other signs for you:
curl -d "name=I%20am%20Ciro" $URL
curl --data-urlencode "name=I am Ciro" $URL
curl ftp://ftp.uk.debian.org/debian/pool/main/[a-z]/
Download:
curl -u ftpuser:ftppass -O ftp://ftp_server/public_html/xss.php
Upload:
curl -u ftpuser:ftppass -T myfile.txt ftp://ftp.testserver.com
curl -u ftpuser:ftppass -T "{file1,file2}" ftp://ftp.testserver.com
Send email:
echo $'sent by curl!\n.' | curl --mail-from user@gmail.com --mail-rcpt user@gmail.com smtp://gmail.com
Body ends with a single dot . on a line.
curl dict://dict.org/show:db #dictionnaries
curl dict://dict.org/d:bash #general
curl dict://dict.org/d:bash:foldoc #computing
Does Basic authentication.
--digest and --ntlm can be used together.
If no :pass, will ask for pass on command line.
Examples:
curl -u user:pass site.with.basic.auth.com
curl --digest -u user:pass site.with.digest.auth.com
Specify proxy server:
curl -x proxysever.test.com:3128
Download iff the file was modified after given date time:
curl -z 01-Jan-00 google.com
Don't decode HTTP specific Content-Encoding and Transfer-Encoding.
E.g.:
printf 'HTTP/1.1 200 OK
Transfer-Encoding: chunked
2
hi
0
' | sed -E 's/$/\r/' | nc -l 8000
Without raw:
curl localhost 8000
we see the decoded body:
hi
With --raw, we see:
2
hi
0
(newline)
(newline)
Get only response for POST: http://stackoverflow.com/questions/10060098/getting-only-response-header-from-http-post-using-curl
Get only status of request: http://superuser.com/questions/272265/getting-curl-to-output-http-status-code
With HEAD it is easy through -I.
Yes by default: http://serverfault.com/questions/199434/how-do-i-make-curl-use-keepalive-from-the-command-line
Seems no, xargs -P it: http://stackoverflow.com/questions/8634109/parallel-download-using-curl-command-line-utility