How to use the “Curl” command in Linux?

The curl command is a versatile tool in the Linux environment, enabling data transfers to and from servers using various protocols including HTTP, HTTPS, FTP, and SMTP. This command-line utility is renowned for its robustness and flexibility, making it essential for sending HTTP requests, retrieving files, uploading data, and testing APIs.

Checking Curl Version

Before using curl, verify its installation and version information using the --version flag. This displays version details, supported protocols, and features.

$ curl --version

curl 7.72.0 (x86_64-pc-linux-gnu) libcurl/7.72.0 OpenSSL/1.1.1g zlib/1.2.11
Release-Date: 2021-02-24
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp smb smbs smtp smtps telnet tftp 
Features: AsynchDNS GSS-API HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL RTSP-CLIENT TLS-SRP UnixSockets

Download Files Using Curl

Single File Download

The -O option downloads a file and preserves its original name from the URL.

$ curl -O http://example.com/file.txt

% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1024  100  1024    0     0   2048      0 --:--:-- --:--:-- --:--:--  2048

Multiple Files Download

Download multiple files simultaneously by specifying multiple -O options:

$ curl -O http://example1.com/file1.html -O http://example2.com/file2.html

Download from URL List

To download files from a list of URLs stored in a text file, use xargs with curl:

$ xargs -n 1 curl -O < urls.txt

% Total % Received % Xferd Average Speed Time Time Time Current
                                 Dload Upload Total Spent Left Speed
100 56100  100 56100    0     0  157k      0 --:--:-- --:--:-- --:--:-- 157k
100 52412  100 52412    0     0  248k      0 --:--:-- --:--:-- --:--:-- 248k

HTTP Headers and Requests

Query HTTP Headers

Use the -I flag to retrieve only HTTP headers without downloading the content:

$ curl -I www.example.com

HTTP/1.1 200 OK
Date: Tue, 07 Feb 2023 00:23:51 GMT
Server: Apache
Last-Modified: Mon, 06 Feb 2022 08:20:07 GMT
ETag: "3d-5a5e5c5dd5e5b"
Accept-Ranges: bytes
Content-Length: 951
Connection: close
Content-Type: text/html; charset=UTF-8

Making POST Requests

Send data using HTTP POST method with the --data option:

$ curl --data "firstName=John&lastName=Doe" https://example.com/api.php

{"status": "success", "message": "Data received: John Doe"}

FTP Operations

Download from FTP Server

Download files from FTP servers with authentication using the -u option:

$ curl -u username:password -O ftp://yourftpserver/yourfile.tar.gz

Upload to FTP Server

Upload files to an FTP server using the -T option:

$ curl -u username:password -T localfile.tar.gz ftp://yourftpserver/

Advanced Features

Cookie Management

Store cookies from a website session to a file for later use:

$ curl --cookie-jar cookies.txt https://www.example.com/login -O

Custom Name Resolution

Override DNS resolution for testing local versions of websites:

$ curl --resolve www.example.com:80:localhost http://www.example.com/

Rate Limiting

Limit download speed to prevent bandwidth saturation:

$ curl --limit-rate 200K http://example.com/largefile.tar.gz -O

Common Use Cases

Conclusion

The curl command is an indispensable tool for Linux users, providing comprehensive functionality for data transfer across multiple protocols. Mastering curl's various options enables efficient file operations, API testing, and web development tasks, making it essential for system administrators and developers alike.