Sending and Fetching Data via HTTP

Many programming tasks involve sending data and receiving data to web services. This is typically done via the HTTP protocol. The XProc step p:http-request can be used to do this in a very declarative way.

The HTTP protocol supports various methods for interacting with data between client and server:

• GET to retrieve data
• POST to send data
• DELETE to delete data
• PATCH to update existing data
• PUT to create new data

For this tutorial, the freely accessible web service https://httpbin.org is used, which simply returns the sent data. This makes it easy to check whether the HTTP request is technically working.

Creating a GET Request

GET is used to request data from a specified resource. It can retrieve data to a client, such as HTML documents, images, and other files: To send a GET request, you need to specify the URL of the resource you want to retrieve with the href option. Frequently, we need to add additional headers, e.g. an API key for authentification. Because p:http-request expects the declaration of an input binding, we just add p:empty.

<?xml version="1.0" encoding="UTF-8"?>
<p:declare-step xmlns:p="http://www.w3.org/ns/xproc" version="3.0">
  
  <p:output port="result"/>
  
  <p:option name="my-api-key" select="'123456'"/>
  
  <p:http-request name="http-request" 
                  method="get"
                  headers="map{
                    'Authorization-key': $my-api-key
                  }"
                  href="https://httpbin.org/get">
    <p:with-input>
      <p:empty/>
    </p:with-input>
  </p:http-request>
  
</p:declare-step>

Output

{
  "args":{},
  "headers":
  {
    "Accept":"*\/*",
    "Accept-Encoding":"gzip,deflate",
    "Authorization-Key":"123456",
    "Host":"httpbin.org",
    "User-Agent":"Apache-HttpClient\/4.5.10 (Java\/18.0.2)",
    "X-Amzn-Trace-Id":"Root=1-66709753-64142de2611472f721b33354"
  },
  "origin":"79.252.159.238",
  "url":"https:\/\/httpbin.org\/get"
}

Performing a POST Request

We learned how to fetch data with XProc’s p:http-request. For submitting a file to a web service, we use the HTTP POST method. Here is a simple example that sends an XML document to a web service. The web service just takes our XML document and returns it in is response:

<?xml version="1.0" encoding="UTF-8"?>
<p:declare-step xmlns:p="http://www.w3.org/ns/xproc" version="3.0">
  
  <p:input port="source">
    <p:inline>
      <root>Our POST test</root>
    </p:inline>
  </p:input>
  
  <p:output port="result"/>
  
  <p:http-request name="http-request" 
                  method="post"
                  href="https://httpbin.org/post">
  </p:http-request>
  
</p:declare-step>

Output

{
  "args":{},
  "data":"<?xml version=\"1.0\" encoding=\"UTF-8\"?><root>Our POST test<\/root>",
  "files":{},
  "form":{},
  "headers":
    {
      "Accept":"*\/*",
      "Accept-Encoding":"gzip,deflate",
      "Content-Length":"77",
      "Content-Type":"application\/xml",
      "Host":"httpbin.org",
      "User-Agent":"Apache-HttpClient\/4.5.10 (Java\/18.0.2)",
      "X-Amzn-Trace-Id":"Root=1-66709350-7e0c1a231cbc663e3aab925c"
    },
    "json":null,
    "origin":"79.252.159.238",
    "url":"https:\/\/httpbin.org\/post"
}

Creating a Multipart Request

Multipart requests combine one or more different sets of data into a single body. Consider you want to upload a photo and add some plain text metadata associated as well, thus making the request containing different types of data (e.g. binary, text), which implies the usage of multipart requests.

The document properties are used in XProc as parts of your message. Only the pre-defined base-uri and serialization properties are not used. If we want to upload our photo alongside its caption and author, we can just use the latter as document properties.

<?xml version="1.0" encoding="UTF-8"?>
<p:declare-step xmlns:p="http://www.w3.org/ns/xproc" version="3.0">
  
  <p:input port="source" content-types="image/png">
    <p:document href="demojam-2024.png"/>
  </p:input>
  
  <p:output port="result"/>
  
  <p:http-request name="http-request"
                  headers="map{
                    'content-type':'multipart/form-data',
                    'author':'Martin Kraetke',
                    'caption':'Me presenting this tutorial at the XML Prague 2024 Demojam'
                  }"
                  method="post"
                  href="https://httpbin.org/post">
  </p:http-request>
  
</p:declare-step>

The server will return the image encoded. In order to save you from reading lots of not human-readable information, I shortened the server result:

Output

{
  "args":{},
  "data":"",
  "files":{},
  "form":
    {
    "null":"?PNG\r\n?\n???\rIHDR???????????????A>?? ?IDATx??Y(...)c@g????IEND?B`?"
    },
    "headers":
      {
        "Accept":"*\/*",
        "Accept-Encoding":"gzip,deflate",
        "Author":"Martin Kraetke",
        "Caption":"Me presenting this tutorial at the XML Prague 2024 Demojam",
        "Content-Length":"33423",
        "Content-Type":"multipart\/form-data; boundary=fe0bf790-ac70-41da-ada6-770a285daf3b",
        "Host":"httpbin.org",
        "User-Agent":"Apache-HttpClient\/4.5.10 (Java\/18.0.2)",
        "X-Amzn-Trace-Id":"Root=1-66708eab-2572b4727df51594284b5b08"
      },
    "json":null,
    "origin":"79.252.159.238",
    "url":"https:\/\/httpbin.org\/post"
}