Question 120005 in Firebase Question posted in
An extensive official documentation can be found here.

Firebase – How do I extract details (body, from, subject) from the incoming SendGrid inbound parse webhook raw email?

I’m receiving the following payload in my SendGrid inbound parse webhook (the webhook points at my firebase cloud function):

data435 --xYzZY
Content-Disposition: form-data; name="dkim"

{@gmail.com : pass}
--xYzZY
Content-Disposition: form-data; name="email"

Received: by mx0099p1iad2.sendgrid.net with SMTP id 0QPMKulr9m Fri, 30 Sep 2022 06:39:28 +0000 (UTC)
Received: from mail-lf1-f47.google.com (unknown [209.85.167.47]) by mx0099p1iad2.sendgrid.net (Postfix) with ESMTPS id 0780EE0961 for <[email protected]>; Fri, 30 Sep 2022 06:39:28 +0000 (UTC)
Received: by mail-lf1-f47.google.com with SMTP id bu25so5446311lfb.3 for <[email protected]>; Thu, 29 Sep 2022 23:39:27 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date; bh=oR2ttmiMxoDGGpj0W7rOOSeDm7QR0gEvVzc1+jKfl08=; b=Eb4zbXHvt4GHz9+gdicE5V+7O7gDB+xXHickuuXb7aoGlIAYIcZS8VArvs3f58VJ9E Z+hM06A63eMHyaVhvaApdRy/e/wVHMzCg4cbjfO8GLxtN+xQ+qwZg6eFO90R8KN7iB4j fDbnA6oemv6kaTwnVrZgrcaVmezBUiH2ZfLykNqtgP9ysx+5aii53EZP1B0qB0Vo3k5r LDUqeo+WyuX3pEVlp9GbE9cVbDpcx8dKCqiO5Hy3BSMMHtspgNIALLpX8JMLMjgsWxbe LV7S3r8NMsfehYaATc38JZZVlRYB/SDQeBF7ythydBxWxJ8onH+l/9yggQSsyKdG+JNf WiMw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date; bh=oR2ttmiMxoDGGpj0W7rOOSeDm7QR0gEvVzc1+jKfl08=; b=Spj3+a3z45RsOcrk946mJ3talAofD9bCbqhHksXvnYg144sSj9h6uFnI7D4e9r7Yrt 9Cs/ptj9x83wZu55/luMtdmBTXGm8XgTJyi1fOzZa6gANBJbf9FHde7avhpCBbpc+udO cMERu0gQSM6shBiO0bOoMpywG07T6qW3glEEog5W/yji1NtZNzgZ7QhhZJAOpD4zwXKe JeLztSey/O8HO6/c41gi6HVX7LYEOztqEzKqXvvcXBkLjCkjZVZh37NcVpzgJ15PoP3V SB1sB25XpCGsbf4+7Rd+cbJ8GjPtbgigfJtqN1vX7z1PAp4EaGyCV/TurruvEph56CL/ OkhA==
X-Gm-Message-State: ACrzQf1LFncmabsqOilbRnxolKJ1KtE0K7WwJYZ3D6ne5hGk9k5qTNxm ge5gg+/XqquwMgtMzBXvkkAhdv0gwQ5UAan7jjqlLcjynnM=
X-Google-Smtp-Source: AMsMyM7irSVbK1v42mqnKCK3s25yqN2XM6XS7q8Ya//jBjOBnWAXlb33VmBIZAQNurF3DqzD4RXYcCvrkb2fztc1nig=
X-Received: by 2002:a05:6512:b85:b0:499:202a:bde9 with SMTP id b5-20020a0565120b8500b00499202abde9mr2950268lfv.550.1664519966548; Thu, 29 Sep 2022 23:39:26 -0700 (PDT)
MIME-Version: 1.0
From: 5WsApp Dev1 <[email protected]>
Date: Fri, 30 Sep 2022 09:39:15 +0300
Message-ID: <CAO57T8mOpdo8eQd9BKiY4cPB0fSEsA=931erdHZOtskUD29QcQ@mail.gmail.com>
Subject: sub1234
To: [email protected]
Content-Type: multipart/alternative; boundary="00000000000053e68705e9df4158"

--00000000000053e68705e9df4158
Content-Type: text/plain; charset="UTF-8"

body1234

--00000000000053e68705e9df4158
Content-Type: text/html; charset="UTF-8"

<div dir="ltr">body1234</div>

--00000000000053e68705e9df4158--

--xYzZY
Content-Disposition: form-data; name="to"

[email protected]
--xYzZY
Content-Disposition: form-data; name="from"

5WsApp Dev1 <[email protected]>
--xYzZY
Content-Disposition: form-data; name="sender_ip"

209.85.167.47
--xYzZY
Content-Disposition: form-data; name="envelope"

{"to":["[email protected]"],"from":"[email protected]"}
--xYzZY
Content-Disposition: form-data; name="subject"

sub1234
--xYzZY
Content-Disposition: form-data; name="charsets"

{"to":"UTF-8","subject":"UTF-8","from":"UTF-8"}
--xYzZY
Content-Disposition: form-data; name="SPF"

pass
--xYzZY--

How do I extract details from it (body, from, to, subject)?
I have tried libraries such "simpleParser" but it just doesn’t extract the details.

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    OK, I found the solution.  I talked to SendGrid support and they told me it doesn't send the payload in JSON format but as base64 that must be converted to string. Therefore, when using Google cloud functions, you need to extract the details from the string.  Make sure you unchecked "POST the raw, full MIME message" at the defined host at  https://app.sendgrid.com/settings/parse 

    I came around the problem the following way:

      const payload = Buffer.from(req.body).toString(); // Convert from base64 to string

  let array = payload.split("--xYzZY") //This is the default boundary for multipart MIME messages

  let body = array.find(t => t.includes("Content-Disposition: form-data; name="text""))
  body = body.split('"text"')[1].trim()

  let to = array.find(t => t.includes("Content-Disposition: form-data; name="to""))
  to = to.split('"to"')[1].trim()

  let from = array.find(t => t.includes("Content-Disposition: form-data; name="from""))
  from = from.split('"from"')[1].trim()

  let attachments = array.find(t => t.includes("Content-Disposition: form-data; name="attachments""))
  attachments = attachments.split('"attachments"')[1]
  let numOfAttach = parseInt(attachments.trim())

  2. 0

    The trick is that you need to parse the parameters that come with the POST request (Content-Type: multipart/form-data). Then, you can easily extract the default parameters you are looking for.
    Here’s how you would do that with fastify

    import fastify, { FastifyInstance } from "fastify";
import fastifyMultipart from "fastify-multipart";

const server: FastifyInstance = fastify({});

server
  .register(fastifyMultipart, { addToBody: true })
  .all("/mail", async (request, reply) => {
    const sgBody = request.body as EmailBody;

    const sender = JSON.parse(sgBody.envelope).from;
   
    // Read the parameters here
   });

server.listen(3000, function (err, address) {
  if (err) {
    server.log.error(err);
    process.exit(1);
  }
  console.log(`Restarted at: ${address}`);
});

    PS: This snippet comes from my sample repo on GitHub.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search