How to use HTTPS outcalls: POST
A minimal example of how to make a POST
HTTP request. The purpose of this dapp is only to show how to make HTTP requests from a canister. It sends a POST
request with some JSON to a free API where you can verify the headers and body were sent correctly.
The HTTPS outcalls feature only works for sending HTTP POST
requests to servers or API endpoints that support IPV6.
In order to verify that your canister sent the HTTP request you expected, this canister is sending HTTP requests to a public API service where the HTTP request can be inspected. As you can see the image below, the POST
request headers and body can be inspected to make sure it is what the canister sent.
Important notes on POST
requests
Because HTTP outcalls go through consensus, a developer should expect any HTTP POST
request from a canister to be sent many times to its destination. Even if you ignore the Web3 component, multiple identical POST requests are not a new problem in HTTPS where it is common for clients to retry requests for a variety of reasons (e.g. destination server being unavailable).
The recommended way for HTTP POST
requests is to add the idempotency keys in the header so the destination server knows which POST
requests from the client are the same.
Developers should be careful that the destination server understands and uses idempotency keys. A canister can be coded to send idempotency keys, but it is ultimately up to the recipient server to know what to do with them. Here is an example of an API service that uses idempotency keys.
POST
example
- Motoko
- Rust
import Blob "mo:base/Blob";
import Cycles "mo:base/ExperimentalCycles";
import Text "mo:base/Text";
import IC "ic:aaaaa-aa";
actor {
//function to transform the response
public query func transform({
context : Blob;
response : IC.http_request_result;
}) : async IC.http_request_result {
{
response with headers = []; // not intersted in the headers
};
};
//PULIC METHOD
//This method sends a POST request to a URL with a free API we can test.
public func send_http_post_request() : async Text {
//1. SETUP ARGUMENTS FOR HTTP GET request
// 1.1 Setup the URL and its query parameters
//This URL is used because it allows us to inspect the HTTP request sent from the canister
let host : Text = "putsreq.com";
let url = "https://putsreq.com/aL1QS5IbaQd4NTqN3a81"; //HTTP that accepts IPV6
// 1.2 prepare headers for the system http_request call
//idempotency keys should be unique so we create a function that generates them.
let idempotency_key : Text = generateUUID();
let request_headers = [
{ name = "Host"; value = host # ":443" },
{ name = "User-Agent"; value = "http_post_sample" },
{ name = "Content-Type"; value = "application/json" },
{ name = "Idempotency-Key"; value = idempotency_key },
];
// The request body is a Blob, so we do the following:
// 1. Write a JSON string
// 2. Convert Text into a Blob
let request_body_json : Text = "{ \"name\" : \"Grogu\", \"force_sensitive\" : \"true\" }";
let request_body = Text.encodeUtf8(request_body_json);
// 1.3 The HTTP request
let http_request : IC.http_request_args = {
url = url;
max_response_bytes = null; //optional for request
headers = request_headers;
//note: type of `body` is ?Blob so we pass it here as "?request_body" instead of "request_body"
body = ?request_body;
method = #post;
transform = ?{
function = transform;
context = Blob.fromArray([]);
};
};
//2. ADD CYCLES TO PAY FOR HTTP REQUEST
//IC management canister will make the HTTP request so it needs cycles
//See: https://internetcomputer.org/docs/current/motoko/main/cycles
//The way Cycles.add() works is that it adds those cycles to the next asynchronous call
//See:
// - https://internetcomputer.org/docs/current/references/ic-interface-spec/#ic-http_request
// - https://internetcomputer.org/docs/current/references/https-outcalls-how-it-works#pricing
// - https://internetcomputer.org/docs/current/developer-docs/gas-cost
Cycles.add<system>(230_850_258_000);
//3. MAKE HTTPS REQUEST AND WAIT FOR RESPONSE
let http_response : IC.http_request_result = await IC.http_request(http_request);
//4. DECODE THE RESPONSE
//As per the type declarations, the BODY in the HTTP response
//comes back as Blob. Type signature:
//public type http_request_result = {
// status : Nat;
// headers : [HttpHeader];
// body : Blob;
// };
//We need to decode that Blob that is the body into readable text.
//To do this, we:
// 1. Use Text.decodeUtf8() method to convert the Blob to a ?Text optional
// 2. We use a switch to explicitly call out both cases of decoding the Blob into ?Text
let decoded_text : Text = switch (Text.decodeUtf8(http_response.body)) {
case (null) { "No value returned" };
case (?y) { y };
};
//5. RETURN RESPONSE OF THE BODY
let result : Text = decoded_text # ". See more info of the request sent at: " # url # "/inspect";
result;
};
//PRIVATE HELPER FUNCTION
//Helper method that generates a Universally Unique Identifier
//this method is used for the Idempotency Key used in the request headers of the POST request.
//For the purposes of this exercise, it returns a constant, but in practice it should return unique identifiers
func generateUUID() : Text {
"UUID-123456789";
};
};
//1. IMPORT IC MANAGEMENT CANISTER
//This includes all methods and types needed
use ic_cdk::api::management_canister::http_request::{
http_request, CanisterHttpRequestArgument, HttpHeader, HttpMethod, HttpResponse, TransformArgs,
TransformContext,
};
use ic_cdk_macros::{self, query, update};
use serde::{Serialize, Deserialize};
use serde_json::{self, Value};
// This struct is legacy code and is not really used in the code.
#[derive(Serialize, Deserialize)]
struct Context {
bucket_start_time_index: usize,
closing_price_index: usize,
}
//Update method using the HTTPS outcalls feature
#[ic_cdk::update]
async fn send_http_post_request() -> String {
//2. SETUP ARGUMENTS FOR HTTP GET request
// 2.1 Setup the URL
let host = "putsreq.com";
let url = "https://putsreq.com/aL1QS5IbaQd4NTqN3a81";
// 2.2 prepare headers for the system http_request call
//Note that `HttpHeader` is declared in line 4
let request_headers = vec![
HttpHeader {
name: "Host".to_string(),
value: format!("{host}:443"),
},
HttpHeader {
name: "User-Agent".to_string(),
value: "demo_HTTP_POST_canister".to_string(),
},
//For the purposes of this exercise, Idempotency-Key" is hard coded, but in practice
//it should be generated via code and unique to each POST request. Common to create helper methods for this
HttpHeader {
name: "Idempotency-Key".to_string(),
value: "UUID-123456789".to_string(),
},
HttpHeader {
name: "Content-Type".to_string(),
value: "application/json".to_string(),
},
];
//note "CanisterHttpRequestArgument" and "HttpMethod" are declared in line 4.
//CanisterHttpRequestArgument has the following types:
// pub struct CanisterHttpRequestArgument {
// pub url: String,
// pub max_response_bytes: Option<u64>,
// pub method: HttpMethod,
// pub headers: Vec<HttpHeader>,
// pub body: Option<Vec<u8>>,
// pub transform: Option<TransformContext>,
// }
//see: https://docs.rs/ic-cdk/latest/ic_cdk/api/management_canister/http_request/struct.CanisterHttpRequestArgument.html
//Where "HttpMethod" has structure:
// pub enum HttpMethod {
// GET,
// POST,
// HEAD,
// }
//See: https://docs.rs/ic-cdk/latest/ic_cdk/api/management_canister/http_request/enum.HttpMethod.html
//Since the body in HTTP request has type Option<Vec<u8>> it needs to look something like this: Some(vec![104, 101, 108, 108, 111]) ("hello" in ASCII)
//where the vector of u8s are the UTF. In order to send JSON via POST we do the following:
//1. Declare a JSON string to send
//2. Convert that JSON string to array of UTF8 (u8)
//3. Wrap that array in an optional
let json_string : String = "{ \"name\" : \"Grogu\", \"force_sensitive\" : \"true\" }".to_string();
//note: here, r#""# is used for raw strings in Rust, which allows you to include characters like " and \ without needing to escape them.
//We could have used "serde_json" as well.
let json_utf8: Vec<u8> = json_string.into_bytes();
let request_body: Option<Vec<u8>> = Some(json_utf8);
// This struct is legacy code and is not really used in the code. Need to be removed in the future
// The "TransformContext" function does need a CONTEXT parameter, but this implementation is not necessary
// the TransformContext(transform, context) below accepts this "context", but it does nothing with it in this implementation.
// bucket_start_time_index and closing_price_index are meaninglesss
let context = Context {
bucket_start_time_index: 0,
closing_price_index: 4,
};
let request = CanisterHttpRequestArgument {
url: url.to_string(),
max_response_bytes: None, //optional for request
method: HttpMethod::POST,
headers: request_headers,
body: request_body,
transform: Some(TransformContext::new(transform, serde_json::to_vec(&context).unwrap())),
// transform: None, //optional for request
};
//3. MAKE HTTPS REQUEST AND WAIT FOR RESPONSE
//Note: in Rust, `http_request()` already sends the cycles needed
//so no need for explicit Cycles.add() as in Motoko
match http_request(request).await {
//4. DECODE AND RETURN THE RESPONSE
//See:https://docs.rs/ic-cdk/latest/ic_cdk/api/management_canister/http_request/struct.HttpResponse.html
Ok((response,)) => {
//if successful, `HttpResponse` has this structure:
// pub struct HttpResponse {
// pub status: Nat,
// pub headers: Vec<HttpHeader>,
// pub body: Vec<u8>,
// }
//We need to decode that Vec<u8> that is the body into readable text.
//To do this, we:
// 1. Call `String::from_utf8()` on response.body
// 3. We use a switch to explicitly call out both cases of decoding the Blob into ?Text
let str_body = String::from_utf8(response.body)
.expect("Transformed response is not UTF-8 encoded.");
ic_cdk::api::print(format!("{:?}", str_body));
//The API response will looks like this:
// { successful: true }
//Return the body as a string and end the method
let result: String = format!(
"{}. See more info of the request sent at: {}/inspect",
str_body, url
);
result
}
Err((r, m)) => {
let message =
format!("The http_request resulted into error. RejectionCode: {r:?}, Error: {m}");
//Return the error as a string and end the method
message
}
}
}
// Strips all data that is not needed from the original response.
#[query]
fn transform(raw: TransformArgs) -> HttpResponse {
let headers = vec![
HttpHeader {
name: "Content-Security-Policy".to_string(),
value: "default-src 'self'".to_string(),
},
HttpHeader {
name: "Referrer-Policy".to_string(),
value: "strict-origin".to_string(),
},
HttpHeader {
name: "Permissions-Policy".to_string(),
value: "geolocation=(self)".to_string(),
},
HttpHeader {
name: "Strict-Transport-Security".to_string(),
value: "max-age=63072000".to_string(),
},
HttpHeader {
name: "X-Frame-Options".to_string(),
value: "DENY".to_string(),
},
HttpHeader {
name: "X-Content-Type-Options".to_string(),
value: "nosniff".to_string(),
},
];
let mut res = HttpResponse {
status: raw.response.status.clone(),
body: raw.response.body.clone(),
headers,
..Default::default()
};
if res.status == 200 {
res.body = raw.response.body;
} else {
ic_cdk::api::print(format!("Received an error from coinbase: err = {:?}", raw));
}
res
}
To use HTTPS outcalls you must update the canister's Candid file:
service : {
"send_http_post_request": () -> (text);
}
Update the Cargo.toml
file to use the correct dependencies:
[package]
name = "send_http_post_backend"
version = "0.1.0"
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[lib]
crate-type = ["cdylib"]
[dependencies]
candid = "0.8.2"
ic-cdk = "0.6.0"
ic-cdk-macros = "0.6.0"
serde = "1.0.152"
serde_json = "1.0.93"
serde_bytes = "0.11.9"
Headers in the response may not always be identical across all nodes that process the request for consensus, causing the result of the call to be "No consensus could be reached." This particular error message can be hard to debug, but one method to resolve this error is to edit the response using the transform function. The transform function is run before consensus, and can be used to remove some headers from the response.
You can see a deployed version of this canister's send_http_post_request
method onchain here: https://a4gq6-oaaaa-aaaab-qaa4q-cai.raw.icp0.io/?id=fm664-jyaaa-aaaap-qbomq-cai.
Additional resources
- Sample code of HTTP
POST
requests in Rust - Sample code of HTTP
POST
requests in Motoko