How to Use Curl Command with Examples

How to use curl command

Back in the mid-1990’s a Swedish programmer named Daniel Stenberg started a project that eventually became what we know as curl today. curl is a command line tool to transfer data to or from a server, using any of the supported protocols (HTTP, FTP, IMAP, POP3, SCP, SFTP, SMTP, TFTP, TELNET, LDAP or FILE). curl is powered by Libcurl. This tool is preferred for automation, since it is designed to work without user interaction and can transfer multiple files at once.

The curl package is pre-installed on most Linux distributions today. The -V or --version options will return the version, and also the supported protocols and features in your current version.

$ curl -V
curl 7.35.0 (x86_64-pc-linux-gnu) libcurl/7.35.0 OpenSSL/1.0.1f zlib/1.2.8 libidn/1.28 librtmp/2.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp smtp smtps telnet tftp 
Features: AsynchDNS GSS-Negotiate IDN IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP 

Install Curl on Ubuntu/Debian

$ sudo apt install curl

Install Curl on CentOS/Fedora

$ sudo yum install curl

Linux curl command

The curl command allows you to download as well as upload data using CLI (the command line) in Linux. This is its syntax:

$ curl [options] [URL]
  • options - The Curl options starting with one or two dashes.
  • URL - URL of the remote server.

Usage of curl

Simplest form that curl can be used is without any option and it will display the resource specified in the [url] to the standard output.

In the following example we will be retrieving the one of the linoxide.com blog pages:

The command above will print the source-code of the linoxide.com blog page we specify in our terminal window.

If you don’t specify the protocol, curl will try to guess the protocol you want to use, and it will default to HTTP.

$ curl https://linoxide.com/linux-how-to/how-to-install-terraform-on-centos-ubuntu/

You can pass the URL as input to the curl command, and redirect the output to a file.

For example:

$ curl https://linoxide.com/linux-how-to/how-to-install-terraform-on-centos-ubuntu/ > install_terraform.html

The URL syntax is protocol dependent and multiple URLs can be written as sets like:

$ curl https://website.{one, two, three}.com

How to make curl use same download file name

if you want, you can force curl to use the same name of the file being downloaded as the local file name. This can be done using the -O command line option.

$ curl -O http://releases.ubuntu.com/18.04/ubuntu-18.04.2-desktop-amd64.iso

Resume an interrupted download

If a download was interrupted for some reason, you can resume it using the -C to resume the download beginning where it left off.

$ curl -C - -O http://releases.ubuntu.com/18.04/ubuntu-18.04.2-desktop-amd64.iso
$ curl -C - -O http://releases.ubuntu.com/18.04/ubuntu-18.04.2-desktop-amd64.iso
** Resuming transfer from byte position 2297856
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0 1901M    0 9842k    0     0   734k      0  0:44:10  0:00:13  0:43:57  339k

How to download multiple files using curl

To download multiple files at once you can use multiple -O flags followed by the URL of the files you want to download.

$ curl -O [URL1] -O [URL2] -O [URL3] ...

URLs with numeric sequence series can be written as:

$ curl ftp://ftp.example.com/file[1-30].jpeg

Curl displays a progress meter during the file download to indicate the transfer rate, amount of data transferred, time left, etc...

$ curl -# -O http://releases.ubuntu.com/18.04/ubuntu-18.04.2-desktop-amd64.iso
###############                                                           20.9%

If you like a progress bar instead of meter, you can use the -# option as in the example above, or –silent if you want to disable it completely.

$ curl -O --silent http://releases.ubuntu.com/18.04/ubuntu-18.04.2-desktop-amd64.iso

How to resolve the moved website issue

Sometimes you may come across an URL that gets errors like "Moved" or "Moved Permanently". This usually happens when the URL redirects to some other URL. For example, google.com redirects to an www URL, so you get an error like the following:

$ curl google.com
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="http://www.google.com/">here</A>.
</BODY></HTML>

To tell curl to follow the redirect, use the -L command line option instead.

$ curl -L google.com

Get the HTTP Headers of a URL

To fetch the HTTP headers of the specified website, use the -I flag:

$ curl -I www.linoxide.com
HTTP/1.1 301 Moved Permanently
Date: Sat, 13 Apr 2019 16:39:03 GMT
Connection: keep-alive
Cache-Control: max-age=3600
Expires: Sat, 13 Apr 2019 17:39:03 GMT
Location: https://www.linoxide.com/
Server: cloudflare
CF-RAY: 4c6edfb8a8437c78-BEG

How to limit the maximum transfer rate

Usage for limiting the data transfer rate is:

$ curl --limit-rate [value] [URL]

The value can be expressed in bytes, kilobytes with the k suffix, megabytes with the m suffix and gigabytes with the g suffix.

The following command will limit the download speed to 1mb:

$ curl --limit-rate 1m -O http://releases.ubuntu.com/18.04/ubuntu-18.04.2-desktop-amd64.iso

Download files from, or upload files to a FTP Server with or without authentication

To access a protected FTP server with the curl command you need to use the -u flag with a username and password.
The following command will list all of the files and directories in the user’s home directory.

$ curl -u USERNAME:PASSWORD ftp://ftp.linoxide.com/

To download the file use:

$ curl -u USERNAME:PASSWORD ftp://ftp.linoxide.com/file.tar.gz

To upload the file to FTP server use the -T flag:

$ curl -T newfile.tar.gz -u USERNAME:PASSWORD ftp://ftp.linoxide.com/

NOTE: If the FTP server allows anonymous logins, you don't need to use -u username:password

That concludes this article. We have learned a brief history of the cURL origins and demonstrated the most used options. The curl command is a versatile utility to download or upload files using the command line.

If you want to learn more about curl visit the Curl Documentation page.

Read Also:

Slavisa Milojkovic 6:49 pm

About Slavisa Milojkovic

Slaviša is a long time Linux user and system administrator. He was PCLinuxOS Gnome distribution developer and maintainer for several years and active opensource supporter.

Author's All Posts
Like to become part of Linoxide Team and contribute tips? Contact us here.

Comments

Your email address will not be published. Required fields are marked *

All comments are subject to moderation.