README
HTTP support for moonbitlang/async.

#Making simple HTTP request

Simple HTTP request can be made in just one line:

///|
#cfg(any(target="native", target="js"))
async test {
let (response, body) = @http.get("https://www.moonbitlang.com")
inspect(response.code, content="200")
assert_true(body.text().has_prefix("<!doctype html>"))
}

You can use use body.text() to get a String (decoded via UTF8) or body.json() for a Json from the response body.

#Generic HTTP client

Sometimes, the simple one-time @http.get etc. is insufficient, for example you need to reuse the same connection for multiple requests, or the request/response body is very large and need to be processed lazily. In this case, you can use the @http.Client type. @http.Client can be created via @http.Client(uri).

The workflow of performing a request with @http.Client is:

  1. initiate the request via client.request(..)
  2. send the request body by using @http.Client as a @io.Writer
  3. complete the request and obtain response header from the server via client.end_request()
  4. read the response body by using @http.Client as a @io.Reader, or use client.read_all() to obtain the whole response body. Yon can also ignore the body via client.skip_response_body()

The helpers client.get(..), client.put(..) etc. can be used to perform step (1)-(3) above.

A complete example:

///|
#cfg(any(target="native", target="js"))
async test {
let client = @http.Client("https://www.moonbitlang.com")
defer client.close()
let response = client..request(Get, "/").end_request()
inspect(response.code, content="200")
let body = client.read_all()
assert_true(body.text().has_prefix("<!doctype html>"))
}

All HTTP client API, including the simple request helpers @http.get etc., are supported on JavaScript backend using fetch API, meaning that you can use these API in browser environment. Notice that some feature, such as proxy, is not supported on JavaScript backend.

#Writing HTTP servers

The recommended way to create HTTP servers is @http.Server::run_forever(..). A HTTP server should first get created via @http.Server(..), after that, server.run_forever(f) automatically start and run the server. The callback function f is used to handle HTTP requests. It receives three parameters:

  • the request to process
  • a &@io.Reader that can be used to read request body
  • a @http.ServerConnection that can be used to send response. The procedure to send a response is:
    1. initiate a response and send response header via .send_response()
    2. send response body by using the @http.ServerConnection as @io.Writer
    3. (optional) complete the response via .end_response(). If .end_response() is not called, it will be called automatically after f returns normally.

Here's an example server that returns 404 to every request:

///|
#cfg(target="native")
pub async fn server(listen_addr : @socket.Addr) -> Unit {
@http.Server(listen_addr).run_forever((request, _body, conn) => {
conn..send_response(404, "NotFound").write("`\{request.path}` not found")
})
}

#
HttpProtocolError

pub suberror HttpProtocolError {
BadRequest
HttpVersionNotSupported(String)
NotImplemented
} derive(ToJson,
Debug
)

#
IncorrectBodyLength

pub suberror IncorrectBodyLength derive(ToJson,
Debug
)

#
ProxyError

pub suberror ProxyError {
ProxyError(Response)
} derive(ToJson,
Debug
)

Error raised when the proxy responded with a non-2XX response code
impl Show for ProxyError

#
URIParseError

pub suberror URIParseError {
InvalidFormat
UnsupportedProtocol(String)
} derive(ToJson,
Debug
)

#
Client

type Client

Simple HTTP client which connect to a remote host via TCP
impl Reader for Client
impl Writer for Client

#
Client::Client

#alias(new, deprecated="`new` is deprecated, use `Client` instead")
async fn Client::Client(uri : String, headers? : Map[String, String], proxy? : Client, verify? : Bool, trust? :
TrustedRoot
) -> Client

Create a new HTTP client by connecting to a remote host. Host should be specified via protocol://host[:port], where protocol is one of http or https. If protocol is https, a TLS connection will be established, and the certificate of the remote peer will be verified.

If the protocol is https, trust will determine the trusted root for cert validation. See @tls.TrustedRoot for more details.

headers can be used to specify persistent headers for the client, i.e. all requests made from this client will share these headers. The ownership of headers will be transferred to the new client, so headers should not be used by the caller later. The following headers is automatically set, and must not be specified in headers:

  • Host
  • Content-Length, Transfer-Encoding

If proxy is present, it should be another HTTP client in a clean state. The new client will send a CONNECT request via the proxy client and try to establish a tunnel via the proxy client. All subsequent requests made by the new client will go through the proxy tunnel. The ownership of the proxy client is transferred to the new client, so it must not be used nor closed anymore by the caller. Using another HTTP client as proxy allows advanced features such as proxy authentication and https CONNECT proxy.

#
Client::close

fn Client::close(self : Client) -> Unit

Close a HTTP client and release underlying resource. In particular close the underlying TCP connection. This function is idempotent: it is safe to call .close() multiple times, only the first .close() call takes effect.

#
Client::end_request

async fn Client::end_request(self : Client) -> Response

End the body of the request currently being sent, and obtain response from the server. Should be called immediately after request body is fully sent.

Only the header of the response will be received and returned, the body of the response can be extracted by using Client as a @io.Reader.

If the body of the last response is still not consumed, it will be discarded.

#
Client::enter_passthrough_mode

async fn Client::enter_passthrough_mode(self : Client) -> Unit

Let the client enter "pass through" mode, where the client serve as a TCP tunnel (maybe TLS encrypted) directly. This is useful for the special CONNECT HTTP request and HTTP protocol upgrade.

In passthrough mode, Read/write the client becomes direct read/write on the underlying connection, and all API except @io.Reader and @io.Writer must not be used anymore.

When entering pass through mode, the client must be in a clean state (i.e. not in the middle of sending a request). Unread data from the body of the last response will be discarded.

#
Client::flush

async fn Client::flush(self : Client) -> Unit

Flush buffered data in the request body being sent, if any.

#
Client::get

async fn Client::get(self : Client, path : String, extra_headers? : Map[String, String], body? : &
Data
) -> Response

Perform a GET request to the server, see Client::request for more details.

#
Client::post

async fn Client::post(self : Client, path : String, body : &
Data
, extra_headers? : Map[String, String]) -> Response

Perform a POST request to the server, see Client::request for more details.

#
Client::put

async fn Client::put(self : Client, path : String, body : &
Data
, extra_headers? : Map[String, String]) -> Response

Perform a PUT request to the server, see Client::request for more details.

#
Client::request

async fn Client::request(self : Client, meth : RequestMethod, path : StringView, extra_headers? : Map[String, String]) -> Unit

Send a HTTP request to the server. Only the header of the request will be sent, request body can be sent by using Client as a @io.Writer. Once request body has been sent, end_request must be called to complete the request and obtain response from the server.

After performing a request, the next request MUST NOT be made before the request is completed via end_request.

In addition to headers in Client::connect, extra HTTP headers can be passed via extra_headers. The following headers is automatically set by request, and must not be specified in extra_headers:

  • Host
  • Transfer-Encoding

If Content-Length is present in extra_headers, it should be the total length of the request body. The request body can still be sent incrementally, but if the actual body being sent is shorter and longer than the provided length, an error will be raised.

#
Client::skip_response_body

async fn Client::skip_response_body(self : Client) -> Unit

Skip the body of the response currently being produced, so that the next request can be made.
pub struct Cookie {
name : String
value : String
path : String?
expires_raw : String?
max_age : Int64?
domain : String?
secure : Bool
http_only : Bool
extensions : Array[String]
} derive(ToJson,
Debug
)

#
Cookie::Cookie

#alias(new, deprecated="`new` is deprecated, use `Cookie` instead")
fn Cookie::Cookie(name : String, value : String, path? : String, expires_raw? : String, max_age? : Int64, domain? : String, secure? : Bool, http_only? : Bool, extensions? : Array[String]) -> Cookie

#
Protocol

pub(all) enum Protocol {
Http
Https
} derive(Compare, Eq, Hash, ToJson,
Debug
)

impl Show for Protocol

#
Protocol::default_port

fn Protocol::default_port(p : Protocol) -> Int

#
Request

pub(all) struct Request {
meth : RequestMethod
path : String
headers : Map[String, String]
} derive(ToJson,
Debug
)

impl Show for Request

#
RequestMethod

pub(all) enum RequestMethod {
Get
Head
Post
Put
Delete
Connect
Options
Trace
Patch
} derive(Compare, Eq, Hash, ToJson,
Debug
)

#
Response

pub(all) struct Response {
code : Int
reason : String
headers : Map[String, String]
cookies : Array[Cookie]
} derive(ToJson,
Debug
)

impl Show for Response

#
Server

pub struct Server {
addr :
Addr

// private fields
}

A HTTP server

#
Server::Server

#alias(new, deprecated="`new` is deprecated, use `Server` instead")
async fn Server::Server(addr :
Addr
, dual_stack? : Bool, reuse_addr? : Bool, headers? : Map[String, String]) -> Server

Create a new HTTP server listening on addr.

The meaning of dual_stack and reuse_addr is the same as @socket.TcpServer::new(), see there for more details.

headers can be used to specify common headers shared by all responses sent by this server.

#
Server::accept

Accept a new connection from a HTTP server. Return the new HTTP connection and the address of peer.

#
Server::addr

#deprecated("use `.addr` instead")
fn Server::addr(self : Server) ->
Addr

Get the listen address of the server

#
Server::close

fn Server::close(self : Server) -> Unit

#
Server::run_forever

async fn Server::run_forever(self : Server, f : async (Request, &
Reader
, ServerConnection) -> Unit, allow_failure? : Bool, max_connections? : Int) -> Unit

Start the main loop of a HTTP server, the server will keep listening for new connections and new requests from existing connections. Requests are handled by the callback function f.

The callback function f accepts three arguments:
  • the request to process
  • a &@io.Reader that can be used to read request body
  • a @http.ServerConnection that can be used to send response. The procedure to send a response is:
    1. initiate a response and send response header via .send_response()
    2. send response body by using the @http.ServerConnection as @io.Writer
    3. (optional) complete the response via .end_response(). If .end_response() is not called, it will be called automatically after f returns.

If allow_failure is true (true by default), error raised by f will not crash the whole server. In this case, run_forever will only terminate when cancelled externally. Note that if f fails, the connection used by f will still get aborted.

If max_connections is present, at most max_connections clients are allowed in parallel. New clients will only get handled after a previous client terminates.

If f fails without closing the connection, run_forever will close each connection automatically. If f manually closes the connection, the connection will be dropped after f returns. On error or cancellation, the server will be closed automatically.

#
ServerConnection

type ServerConnection

A single HTTP server connection

#
ServerConnection::client_addr

Get the address of client from a HTTP server connection

#
ServerConnection::close

fn ServerConnection::close(self : ServerConnection) -> Unit

Close the connection, the underlying TCP connection will be closed as well. This function is idempotent: it is safe to call .close() multiple times, only the first .close() call takes effect.

#
ServerConnection::end_response

async fn ServerConnection::end_response(self : ServerConnection) -> Unit

End the body of the response currently being sent. Should be called immediately after response body is fully sent.

#
ServerConnection::enter_passthrough_mode

async fn ServerConnection::enter_passthrough_mode(self : ServerConnection) -> Unit

Let the server enter "pass through" mode, where the server serve as a TCP tunnel (maybe TLS encrypted) directly. This is useful for handling the special CONNECT HTTP request and HTTP protcol upgrade.

In passthrough mode, Read/write the server becomes direct read/write on the underlying connection, and all API except @io.Reader and @io.Writer must not be used anymore.

When entering pass through mode, the server must be in a clean state (i.e. not in the middle of sending a response). Unread data from the body of the last request will be discarded.

#
ServerConnection::flush

async fn ServerConnection::flush(self : ServerConnection) -> Unit

Flush buffered data in the response body being sent, if any.

#
ServerConnection::new

fn ServerConnection::new(conn :
Tcp
, headers? : Map[String, String]) -> ServerConnection

Create a new HTTP server connection from a TCP connection.

headers can be used to specify persistent headers for the connection, i.e. all response sent by this connection will share these headers. The following headers is automatically set, and must not be specified in headers:

  • Content-Length, Transfer-Encoding

#
ServerConnection::read_request

async fn ServerConnection::read_request(self : ServerConnection) -> Request

Read a single request from the connection. If the body of the last request is not consumed yet, it will be discarded.

After calling read_request, the body of the request can be obtained by using ServerConnection as a @io.Reader.

#
ServerConnection::send_response

async fn ServerConnection::send_response(self : ServerConnection, code : Int, reason : String, extra_headers? : Map[String, String], cookies? : Array[Cookie]) -> Unit

Send a response to the peer. The ServerConnection can be used as a @io.Writer later for sending response body. After calling send_response, end_response must be called before sending the next response.

cookies, if present, specify a list of cookies that the server wish the client to set. They will be transferred to the client via the Set-Cookie HTTP header. Users should always set cookies via cookies instead of filling Set-Cookie header directly.

If Content-Length is present in extra_headers, it should be the total length of the request body. This useful for, for example, file servers to allow clients to know the progress of download, perform multipart download etc.

The request body can still be sent incrementally, but if the actual body being sent is shorter and longer than the provided length, an error will be raised.

#
ServerConnection::skip_request_body

async fn ServerConnection::skip_request_body(self : ServerConnection) -> Unit

Manually discard the body of the request currently being processed.

#
ServerConnection::write_string

async fn ServerConnection::write_string(self : ServerConnection, s : String) -> Unit

Introduced so that the sugar writer <+ "hello \{world}" can work on ServerConnection. Performance will be improved in the future

#
ServerConnection::write_string_interpolation

async fn ServerConnection::write_string_interpolation(self : ServerConnection, data : &
Data
) -> Unit

#
get

async fn get(uri : String, headers? : Map[String, String], body? : &
Data
, proxy? : Client) -> (Response, &
Data
)

Perform a HTTP GET request to uri. Supported protocols are http:// and https://. The HTTP response message and the whole response body will be returned.

proxy, if present, specifies the proxy to use for this request. See @http.Client::new for more details. proxy is not supported on JavaScript backend.

See Client::request for more details.

#
get_stream

async fn get_stream(uri : String, headers? : Map[String, String], body? : &
Data
, proxy? : Client) -> (Response, Client)

Similar to @http.get, but allow reading response body streamingly. A pair (response, client) will be returned, where response is the response header from the server, and client is the HTTP client that performs the request. client can be used to read the content of response body via @io.Reader, see @http.Client for more details.

Note that the returned client must be manually closed via .close() to close the underlying connection used for the request.

#
post

async fn post(uri : String, content : &
Data
, headers? : Map[String, String], proxy? : Client) -> (Response, &
Data
)

Similar to get, but performs a POST request instead.

#
post_stream

async fn post_stream(uri : String, headers? : Map[String, String], proxy? : Client) -> Client

Similar to @http.post, but allow writing request body streamingly. The return value client is the HTTP client that performs the request, it can be used to write the content of request body via @io.Writer. Notice that writing to @http.Client is buffered, so if you need to send data to the server immediately, .flush() must be called. After writing all the content, .end_request() must be called to complete the request and obtain response from the server. After that, the response body from the server can be obtained by using client as a @io.Reader. See @http.Client for more details.

Note that the returned client must be manually closed via .close() to close the underlying connection used for the request.

#
put

async fn put(uri : String, content : &
Data
, headers? : Map[String, String], proxy? : Client) -> (Response, &
Data
)

Similar to get, but performs a PUT request instead.

#
put_stream

async fn put_stream(uri : String, headers? : Map[String, String], proxy? : Client) -> Client

Similar to @http.put, but allow writing request body streamingly. The return value client is the HTTP client that performs the request, it can be used to write the content of request body via @io.Writer. Notice that writing to @http.Client is buffered, so if you need to send data to the server immediately, .flush() must be called. After writing all the content, .end_request() must be called to complete the request and obtain response from the server. After that, the response body from the server can be obtained by using client as a @io.Reader. See @http.Client for more details.

Note that the returned client must be manually closed via .close() to close the underlying connection used for the request.

#
request

async fn request(uri : String, meth : RequestMethod, headers : Map[String, String], body : &
Data
, proxy? : Client) -> (Response, &
Data
)

Perform a single HTTP request to uri. Supported protocols are http:// and https://. The HTTP response message and the whole response body will be returned.

proxy, if present, specifies the proxy to use for this request. See @http.Client::new for more details. proxy is not supported on JavaScript backend.

See Client::request for more details.