Intoduction to AtomPub

Command Line - Curl

You should be comfortable with HTTP and REST. For a quick introduction go here.

Curl is a great tool for testing from the command line. Users have the ability to trace headers being sent and received, and everything in between. Lets first start by checking if you have curl, and if you don't then how you can install it. You can check your version of curl from the command line like so:

joe[~]$ curl --version
curl 7.16.3 (powerpc-apple-darwin9.0) libcurl/7.16.3 OpenSSL/0.9.7l zlib/1.2.3
Protocols: tftp ftp telnet dict ldap http file https ftps
Features: GSS-Negotiate IPv6 Largefile NTLM SSL libz

Where to Get Curl

If the console said -bash: curl: command not found then you will need to install curl. Its relatively simple, just download it from http://curl.haxx.se/, then follow the instructions for your operating system.

General Usage

The best way to get information is man curl. But the man page for curl is quite long. I have not had to use all the options available to curl, but there are some that you will need to know. Here is a list of the most basic and helpful command line switches:

Switch	Meaning
-X HTTP_METHOD --request	Provide an HTTP method such as DELETE, GET, POST, or PUT. If this is not provided the default is GET.
-H HEADER --header	Provide a Header such as "Content-type: application/atom+xml"
-d @/path/to/file --data	Send data with a POST or PUT request. The data will be located inside the provided file. The '@' is required.
-L --location	This will follow 301 redirects or Location headers if they are returned from the server.
-k --insecure	Allow insecure SSL connections. For example: sites with invalid https certificates.
-u USER:PASS --user	Username and password for Basic Authentication. The ':' is required.
-v --verbose	Verbose output. Great for seeing exactly what was sent and received.
-o FILE --output	Send the output to a file instead of Standard out. Very useful if you want to log or debug the output.

Examples

I'll show a quick example of each type of HTTP request. and try to explain what each switch does. The examples are going to be sending requests with Basic Authentication.

Also be aware that because the commands are on multiple lines I am escaping newlines with \ characters. These are not required if you put everything on one line.

GET Request

A GET could be as simple as curl "http://www.google.com" but by providing a few command line options we can make far more powerful requests.

joe[~]$ curl "http://example.com/atom/api" \
             -L -k -v -u joe:p4ssw0rd      \
             -o servicedocument.txt

> GET /atom/api HTTP/1.1
> Authorization: Basic am9lOnA0c3N3MHJk
> User-Agent: curl/7.16.3 (powerpc-apple-darwin9.0)
> Host: example.com
> Accept: */*
...

This shows just how easy using curl is. This curl command does the following:

1. I'm pulling from http://example.com/atom/api

2. Using username joe and password p4ssw0rd

3. The output is saved to servicedocument.txt in the current directory

4. -L or --location allows following redirects

5. -k or --insecure allows trusting an insecure SSL connection

6. -v or --verbose enables verbose output

POST Request

Here we actually have to send data. I'm going to put some XML data into a file and use curl's options to specify that file as the data being sent. In this case I'm going to send an Atom Entry so you'll see that I need to specify the Content-type header.

The data being sent is stored in a file and referenced by the --data option. That data is sent with the request as the HTTP message body. This is how HTTP Requests work in the HTTP Protocol. There must be a blank line between the HTTP Request's headers and message body. So anything after that blank line (the black text below) would be the contents of the file given by the --data option.

joe[~]$ curl "https://example.com/atom/api/createblog"       \
             -X POST -H "Content-type: application/atom+xml" \
             -v -L -k -u joe:p4ssw0rd                        \
             --data @/Users/joe/post_atom_entry.xml          \
             -o post_response.txt

> POST /atom/api/createblog HTTP/1.1
> Authorization: Basic am9lOnA0c3N3MHJk
> User-Agent: curl/7.16.3 (powerpc-apple-darwin9.0)
> Host: example.com
> Accept: */*
> Content-type: application/atom+xml
> Content-Length: 400

<?xml version="1.0" encoding="UTF-8"?>
<entry xmlns="http://www.w3.org/2005/Atom"
       xmlns:snx="http://www.ibm.com/xmlns/prod/sn">
	<id>ignored</id>
	<title type="text">Joe - API Created Blog Entry</title>
  <author><name>Joseph Pecoraro</name></author>
  <content type="text>Nothing major here</content>
</entry>
...

The above shows the curl command, and both the headers and data being sent. Here is each section broken down:

1. I'm sending data to https://example.com/atom/api/createblog

2. Using username joe and password p4ssw0rd

3. The data being sent was stored in /Users/joe/post_atom_entry.xml specified by the --data option. It contained the data shown in black below, being sent. The '@' character means load the data from the file.

4. The output is saved to post_response.txt in the current directory

5. -L or --location allows following redirects

6. -k or --insecure allows trusting an insecure SSL connection

7. -v or --verbose enables verbose output

PUT Request

This is basically the same as a POST except it would likely be sent to a different URL and it would have -X PUT instead of POST. Because they are so similiar, I'm not going to give an example here.

DELETE Request

joe[~]$ curl "https://example.com/atom/api/deleteblog" \
             -X DELETE -L -k -v                        \
             -u joe:p4ssw0rd                           \
             -o response.txt

> DELETE /atom/api/deleteblog HTTP/1.1
> Authorization: Basic am9lOnA0c3N3MHJk
> User-Agent: curl/7.16.3 (powerpc-apple-darwin9.0)
> Host: example.com
> Accept: */*
...

1. I'm sending the request to http://example.com/atom/api/deleteblog

2. Using username joe and password p4ssw0rd

3. The output is saved to response.txt in the current directory

4. -L or --location allows following redirects

5. -k or --insecure allows trusting an insecure SSL connection

6. -v or --verbose enables verbose output

Advanced Switches

Here is a reference of some more switches, again in an easy to understand chart. Its less likely that you'll need to used these, but its still nice to know them in case one day you need to use them.

Switch	Meaning
-A AGENT_STRING --user-agent	Give anything you want as your User Agent header. Great for acting like any browser. This is the User-Agent header.
-b NAME=DATA --cookie	Provide cookie data as a name value pair. Multiple pairs can be expressed like "NAME=DATA; NAME2=DATA". This is the Cookie header.
--compressed	If you have libcurl this option will allow gzipping automatically. This is the Accept-Encoding header.
-D FILE --dump-header	Write the protocol headers to the given FILE. Nice for debugging the headers.
-e URL --referer	Specify a referrer URL. This is the Referer header.
-m SECONDS --max-time	Similiar to the --connect-timeout option but this even breaks on data transfers after the given number of seconds.
-F NAME=DATA --form	Curl acts like it is submitting a form on a webpage with the proper Content-type header. Name and value pairs are sent just like the -b option.
-s --silent	Silent mode. Curl won't show the progress meter or error messages. This can be useful when piping output.
--trace FILE	A full trace dump of incoming and outgoing data will be written to the given file. "-" means stdout.
-w FORMAT --write-out	Provide special options to print out statistics and information after a successful operation has finished. See the man page for details.

Alternatives to Curl - Wget

Many people have switched from curl to Wget. Essentially they do the same things, but there are some differences. The differences are listed here.

Here are even more alternatives and a comparison of each.