How to use the “Curl” command in Linux?

The 'curl' command is a versatile tool in the Linux environment, enabling data transfers to and from servers utilizing a variety of protocols including HTTP, FTP, and SMTP. This command-line utility is renowned for its robustness and versatility, as it can be utilized for various tasks, including sending HTTP requests, retrieving files, and uploading data. Join us as we delve into the fundamentals of utilizing the 'curl' command in Linux, comprehending its varied options and applications.

1. To Check the Curl Version

The "curl --version" command in the Linux operating system displays key information about the version of the 'curl' tool installed on your device. Utilizing the "--version" flag, this command showcases the version number, build date, and other relevant details regarding the 'curl' software. This information proves valuable when diagnosing issues or confirming the compatibility of 'curl' with other software or scripts.

Input

$ curl -version

Output

curl 7.72.0 (x86_64-pc-linux-gnu) libcurl/7.72.0 OpenSSL/1.1.1g zlib/1.2.11 libidn2/2.3.0 libpsl/0.21.0 (+libidn2/2.3.0) nghttp2/1.41.0 librtmp/2.3
Release-Date: 2021-02-24
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp smb smbs smtp smtps telnet tftp 
Features: AsynchDNS GSS-API HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL RTSP-CLIENT SSPI TLS-SRP UnixSockets 

2. Download a File Using Curl Command

The "curl -O" Linux command allows you to download a file from a specified URL and preserve its original name. The "-O" option instructs curl to retain the original name of the remote file when saving it to your local machine. This feature is handy for downloading files from a server and keeping their names unchanged. For instance, if you wanted to download the file "file.txt" from "http://example.com/file.txt", you would simply run "curl -O http://example.com/file.txt" and the file would be saved with the same name, "file.txt", on your local machine.

The result of executing "curl -O" in Linux is contingent on the URL being downloaded. If the URL is correct and the file exists, curl will initiate the download and save the file in the current directory, retaining its original name from the remote server. The terminal will showcase the download progress, including details such as the file's overall size and the amount of data already retrieved.

An example of this output can be seen in the “curl -O http://example.com/file.txt" command, where the downloaded file will be saved as "file.txt" in the present working directory.

Input

$ curl -O

Output

% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   100    0     0  100   100      0      0  0:00:01  0:00:01 --:--:--     0

3. Download Multiple Files using the Curl Command

By executing the following command, you can simultaneously obtain file1.html and file2.html from the sources http://example1.com and http://example2.com, respectively.

$ curl -O http://file1.com/example1.html -O http://example2.com/file2.html 

4. Download URL from a File using Curl

The "xargs -n 1 curl -O < urls.txt" command downloads multiple files efficiently in Linux. The "urls.txt" file contains a list of URLs, and "xargs" passes each line as an argument to "curl", which downloads the files using their original names in the present working directory. The "-n 1" option limits the number of lines passed, and "-O" saves the files. The command shows the download progress and retrieves all files successfully. In short, this command is a simple way to download multiple files in Linux using "curl" and "xargs".

Input

$ xargs -n 1 curl -O < urls.txt

Output

% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 56100 0 56100 0 0 157700 0 --:--:-- --:--:-- --:--:-- 157700
100 52412 0 52412 0 0 248300 0 --:--:-- --:--:-- --:--:-- 248300
100 48976 0 48976 0 0 157700 0 --:--:-- --:--:-- --:--:-- 157700
...

5. Query HTTP Headers

In the Linux environment, the "$ curl -I www.example.com" command serves the purpose of obtaining the HTTP headers of a website located at "www.example.com". The "-I" switch in the curl command functions to only retrieve the HTTP headers, excluding the actual content of the website. These headers include details like the website's status code, date, server, and content type among others, making them useful for activities such as website status checking, debugging, and understanding the website setup.

Input

$ curl -I www.example.com

Output

HTTP/1.1 200 OK
Date: Tue, 07 Feb 2023 00:23:51 GMT
Server: Apache
Last-Modified: Mon, 06 Feb 2022 08:20:07 GMT
ETag: "3d-5a5e5c5dd5e5b"
Accept-Ranges: bytes
Content-Length: 951
Connection: close
Content-Type: text/html; charset=UTF-8

6. With Parameters Make a Post Request using the Curl Command

The "$ curl --data" Linux command serves the purpose of transmitting data to a designated URL. In the provided example, the data being transferred consists of two parameters, "firstName" set to "John" and "lastName" set to "Doe". The data is sent to the URL "https://exampledomain.com/file.php" through the HTTP POST method. This command is frequently used to submit information to websites or web applications for processing. The "--data" flag gives the ability to specify the data that needs to be transmitted, while the "curl" command initiates the transfer.

Input

$ curl --data "firstName=John&lastName=Doe" https://exampledomain.com/file.php

Output

The output for this is based on the specific server and the data sent from HTTP post method, it could be plain text like the following −

My name is John Doe

7. Upload and Download Files from the FTP Server with or without Authentication

Obtaining a file from a remote FTP server is easily accomplished using the proper command. In the scenario where the FTP server is accessible at ftp://server, the following terminal command will download the file "file.tar.gz" and place it in the present working directory.

$ curl -u username:password -O ftp://yourftpserver/yourfile.tar.gz 

Uploading a file named 'file.tar.gz' from your local machine to an FTP server at 'ftp://server' can be achieved with the following curl command

$ curl -u username:password -T mylocalfile.tar.gz ftp://yourftpserver

8. Store Website Cookies using the Curl Command

To view the cookies saved on your computer when browsing to https://www.example.com, utilize the following curl command to download them to a file named 'examplecookies.txt'. Then, you can simply use the cat command to display the contents of the file.

Input

$ curl --cookie-jar examplecookies.txt https://www.example.com/file.html -O

Output

 % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1701k  100 1701k    0     0   951k      0  0:00:01  0:00:01 --:--:--  951k

9. Modify Name Resolution using Curl Command

As a web developer, if you wish to preview the local version of 'demodomain.com' before releasing it to the public, you can use curl to route requests for 'http://www.demodomain.com' to your localhost using the following method −

$ curl --resolve www.demodomain.com:80:localhost http://www.demodomain.com/

10. Limit the Download Rate using the Curl Command

Restrict curl from consuming excessive bandwidth by capping the download speed at 200 KB/s using the following method.

$ curl --limit-rate 200K http://demodomain.com/file.tar.gz -O

Conclusion

To wrap up, the 'curl' command is a significant tool in the Linux landscape, allowing for the seamless transfer of data between servers using a variety of protocols. From sending HTTP requests to downloading files and uploading data, this versatile command provides ample opportunities to streamline your work with web-based information. Acquiring a mastery of the 'curl' command and its different options is a valuable asset, elevating your proficiency in manipulating data in the Linux environment. Whether you are an experienced Linux user or just beginning your journey, investing time to learn the 'curl' command is guaranteed to broaden your horizons and unlock new opportunities.