What is a webhook and how does email to webhook work?

In this article we'll discuss what Webhooks are and how to it's possible to use them for both sending and receiving email. If you already know what a Webhook / HTTP Callback is you can skip to the section on email webhooks.

What is a webhook?

A webhook is a method that allows two different HTTP Servers to communicate with each other. Generally webhooks occur in response to an event in one system that is reported to the other HTTP Server. When server A receives an event it makes an HTTP Post to Server B with a number of details about the event. Webhooks allow you to get real-time updates without a permanent TCP connection to another service. If you already know what a webhook is and just want to know how webhooks relate to email, you can skip the next section.

Creating a webhook endpoint is a similar process to creating any other webpage on your website. It’s an HTTP or HTTPS endpoint on your server with a URL. However, in order for the remote server to contact your website the page must be publicly accessible. Once your URL is accessible you can register it as a callback URL.

What are some examples of webhooks?

Webhooks are a way for one service to tell another service that something has happened. It basically allows you to push data from your app or website and have it sent somewhere else. For example:

Your billing system may use a webhook to inform you of the status of a payment. Long running tasks might let you know that they've finished by posting to a URL on your site. Your monitoring or alerting tool might fire a webhook if your site is unreachable, allowing you to take automated action. Webhooks are a common way to interact with the Slack Messaging API. Webhooks can take a number of different forms. A common format for event data is JSON. However, you may also see webhooks in multipart/form-data format, XML or any other HTTP format. What's important is that real time events trigger webhook requests.

Lets take a deeper look at the webhooks used by Stripe for payments. Whenever an action takes place in stripe, you can register to receive a webhook with the event data. Taking credit card data is complicated, so we may hand off the whole process to a payment provider like Stripe, but how do we know that this was successful?

Well, that's where webhooks come in. Once the customer has completed payment, Stripe can make an HTTP POST to your website with the details. Stripe supports a huge number of webhook event types. However, in this case it might have been the payment_intent.succeeded event.

This event contains all of the expected event data such as the customer, amount and card details so you can confirm the status of the transaction.

Email webhooks

One of the uses for webhooks is to provide email to webhook. Email is transmitted over SMTP and so it's very hard to get information from the SMTP server to your website or application. This is where CloudMailin comes in. CloudMailin allows you to send and receive email via HTTP API.

Receiving email via HTTP POST; how does email to webhook work?

The core of CloudMailin is the ability to receive email via webhook, making an HTTP POST whenever an email is received. When you sign up with CloudMailin, you are provided with an email address, or you can use your own domain and direct every email on that domain through the CloudMailin SMTP servers. When CloudMailin receives an email for your registered address, you'll get a webhook message event containing the content of the email and all of the associated metadata.

There's no need to parse the email as the webhook event will contain all of the information in an easy-to-use format designed for your application.

CloudMailin has several different HTTP POST formats which make integration easier.

When you receive an email the message looks something like this:

{
  "headers": {
    "return_path": "from@example.com",
    "received": [
      "by 10.52.90.229 with SMTP id bz5cs75582vdb; Mon, 16 Jan 2012 09:00:07 -0800",
    ],
    "from": "Message Sender <sender@example.com>",
    "to": "Message Recipient<to@example.co.uk>",
    "message_id": "<4F145791.8040802@example.com>",
    "subject": "Test Subject",
  "envelope": {
    //...
  },
  "plain": "Test with HTML.",
  "html": "<html><head>\n<meta http-equiv=\"content-type\" content=\"text/html; charset=ISO-8859-1\"></head><body\n bgcolor=\"#FFFFFF\" text=\"#000000\">\nTest with <span style=\"font-weight: bold;\">HTML</span>.<br>\n</body>\n</html>",
  "reply_plain": "Message reply if found.",
  "attachments": []
}

We've shortened the JSON above for the purpose of this article but you can see the Full Example in the documentation.

Forwarding email to webhooks

Since you're provided with an email address it's also possible to forward email to webhooks with CloudMailin. This would allow you to relay email from other services such as Gmail to a webhook too. To do this, you need to set up a regular email forward from within your Gmail or other service account to your Cloudmailin address creating an easy Gmail to webhook.

Webhooks when sending emails

It's also possible to use webhooks when sending emails. All message events can be sent as a callback to let your application know the status of your outbound email delivery. Email delivery is complicated and it's important to be informed of the status of your message deliverability. For example, registering a bounce webhook allows you to curate your customer list and make sure that you're not sending messages that bounce. Importantly, it also allows you to give the customer the opportunity to update the bounced email address to ensure they can be reached.

CloudMailin allows you to register a webhook to get a callback for all email message events when sending email.

What is the alternative to webhooks?

Webhooks are a hugely powerful tool, especially when the two systems communicating wouldn't normally be compatible. However, there are drawbacks. The main drawback is that you need to able to receive the webhook and that needs a publicly accessible HTTP Server.

The alternative is to open a connection directly from your website to a remote server and receive a bi-directional flow of information. Creating a TCP connection from a website can be challenging but long-polling or websockets can be utilised for this. For example, the messaging app Telegram uses long-polling to receive messages in node rather than webhooks.

Using websockets or long-polling on the server side can be complex though so it's often easier to deal with webhooks.

It's also possible that a server might not be available when an event occurs. Webhooks are straight forward here because they can simply retry later.

CloudMailin will retry any emails whenever a 5xx status is received in response to a callback event

Working with webhooks

As we mentioned above, working with webhooks requires a publicly accessible server. This doesn't lend itself to easy development. However, there are a couple of options:

Local development with webhooks

In order to make developing with webhooks easier we developed a tool called WebhookApp. Although it was designed by CloudMailin to receive email data, WebhookApp allows you to receive any webhook event, capture the event and display it on a page. You can then replay the same POST locally or relay the webhook.

Another great tool is Postman. Postman is a tool you can run locally to make HTTP requests. Using Postman you can simulate the callback that your website would receive.

CloudMailin makes several Postman requests available to develop locally.

It's also possible to use tools like Ngrok, which gives you a publicly accessible URL to relay your webhook to your local development server.

Webhook Security

Finally, it's worth mentioning security. It's often desirable to a) make sure that your content is transmitted securely and b) make sure that the service making the HTTP POST to your site is who you expect.

The first problem can be solved with HTTPS and TLS. Using HTTP encryption ensures that your webhook data cannot be spied upon by malicious actors.

CloudMailin highly recommends that HTTP is used when receiving email via webhook or receiving delivery status events for inbound emails. We also recommend using HTTPS for any email relay events or bounce events when sending.

In order to ensure that your Callback URL or Target URL can only be reached by the intended system you should also secure the endpoint. The simplest way to secure an HTTPS endpoint is with basic authentication, but other schemes like Bearer tokens work just as well.

Summary

In this article we've covered what webhooks are, and specifically what an email webhook is, as well as how you can use and secure webhooks. We've also covered how CloudMailin can help you to get started with webhooks for SMTP.

If you want to get started sending and receiving email with your web application, we'd love it if you give CloudMailin a try.