AWS Lambda Receive Email Forward Function

aws_lambda_forward_email_inline.py

from __future__ import print_function
import boto3
import json
import os
import re
from email.parser import Parser
from datetime import datetime
import logging

## Thanks to:
# https://gist.github.com/stenius/c6983f990bbbb1e49e4f
# https://bravokeyl.com/how-to-set-up-email-forwarding-with-amazon-ses/#Create-a-Lambda-function-to-forward-recieved-email
# https://github.com/arithmetric/aws-lambda-ses-forwarder
###
# Dincer Kavraal -- dincer(AT)mctdata.com
###


Log = logging.getLogger()
Log.setLevel(logging.DEBUG)

FORWARD_ADDRESSES = ["to-me@example.com",
		     "to-him-1@example.com",
		     "to-her-2@example.com"]
FROM_ADDRESS = "Sender <example@example.com>"
EMAIL_DOMAIN = "example.com"  # my aws receiver email domain
SPAMMER_DOMAINS = map(re.compile, ["example.com"])
SPAMMER_EMAILS = map(re.compile, ["spammer-jack@example.com"])


RE_DOMAIN = re.compile("\@(.*)$")

def decode_email(msg_str):
	p = Parser()
	message = p.parsestr(msg_str)
	decoded_message = ''
	for part in message.walk():
		charset = part.get_content_charset()
		if part.get_content_type() == 'text/plain':
			part_str = part.get_payload(decode=1)
			decoded_message += part_str.decode(charset)
	return decoded_message
	
def print_with_timestamp(*args):
	print(datetime.utcnow().isoformat(), *args)

def lambda_handler(event, context):
	print_with_timestamp('Starting - inbound-sns-spam-filter')
	
	Log.debug(json.dumps(event, indent=4))

	ses_notification = event['Records'][0]['Sns']
	message_id = ses_notification['MessageId']
	message = json.loads(ses_notification["Message"])
	receipt = message['receipt']
	sender = message['mail']['source']
	subject = message['mail']['commonHeaders']['subject']
	sender_domain = (RE_DOMAIN.findall(sender) or [""])[0]
	
	print_with_timestamp('Processing message:', message_id)

	# Check if any spam check failed
	if (receipt['spfVerdict']['status'] == 'FAIL' or
			receipt['dkimVerdict']['status'] == 'FAIL' or
			receipt['spamVerdict']['status'] == 'FAIL' or
			receipt['virusVerdict']['status'] == 'FAIL' or
			all(map(lambda x: x.search(sender_domain), SPAMMER_DOMAINS)) or
			all(map(lambda x: x.search(sender), SPAMMER_EMAILS))):

		send_bounce_params = {
			'OriginalMessageId': message_id,
			'BounceSender': 'mailer-daemon@{}'.format(EMAIL_DOMAIN),
			'MessageDsn': {
				'ReportingMta': 'dns; {}'.format(EMAIL_DOMAIN),
				'ArrivalDate': datetime.now().isoformat()
			},
			'BouncedRecipientInfoList': []
		}

		for recipient in receipt['recipients']:
			send_bounce_params['BouncedRecipientInfoList'].append({
				'Recipient': recipient,
				'BounceType': 'ContentRejected'
			})

		print_with_timestamp('Bouncing message with parameters:')
		print_with_timestamp(json.dumps(send_bounce_params))

		try:
			ses_client = boto3.client('ses')
			bounceResponse = ses_client.send_bounce(**send_bounce_params)
			print_with_timestamp('Bounce for message ', message_id, ' sent, bounce message ID: ', bounceResponse['MessageId'])
			return {'disposition': 'stop_rule_set'}
		except Exception as e:
			print_with_timestamp(e)
			print_with_timestamp('An error occurred while sending bounce for message: ', message_id)
			raise e
	else:
		print_with_timestamp('Accepting message:', message_id)

	# now distribute to list:
	action = receipt['action']
	if (action['type'] != "S3"):
		Log.exception("Mail body is not saved to S3. Or I have done sth wrong.")
		return None
	
	try:
		ses_client = boto3.client('ses')
		s3_client = boto3.resource('s3')
		mail_obj = s3_client.Object(action['bucketName'], action['objectKey'])
		body = decode_email(mail_obj.get()["Body"].read())
		try:
			response = ses_client.send_email(
				Source=FROM_ADDRESS,
				Destination={
					'ToAddresses': FORWARD_ADDRESSES,
				},  
				Message={
					'Subject': {
						'Data': subject,
					},
					'Body': {
						'Text': {
							'Data': body,
						},
						'Html': {
							'Data': body.replace("\n", "<br />"),
						}
					}
				},  
			)
		except Exception as e1:
			print_with_timestamp(e1)
			print_with_timestamp('An error occurred while sending bounce for message: ', message_id)
			raise e1
	except Exception as e2:
		print_with_timestamp(e2)
		print_with_timestamp('An error occurred while sending bounce for message: ', message_id)
		raise e2

	return None
	
	

Setting_Up_Email_Receiving_in_AWS_Lambda_and_Forwarding.md

Set up an IAM Role called (say) SNSEmailForwarder:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": "arn:aws:logs:*:*:*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "sns:GetTopicAttributes",
                "sns:List*",
                "sns:Publish"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "ses:SendRawEmail",
                "ses:SendEmail",
                "ses:SendBounce"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:PutObject"
            ],
            "Resource": "arn:aws:s3:::YOUR_S3_BUCKET/*"
        }
    ]
}

Verify your email domain on AWS SES (you can't use sandboxed mail domain, so says AWS) :

1. http://docs.aws.amazon.com/ses/latest/DeveloperGuide/verify-domains.html

Setup an SNS topic:

1. Go to https://console.aws.amazon.com/sns
2. Create topic
3. Topic Name: SNSForwardEmails

Setup a Lambda function: 0) Go to https://console.aws.amazon.com/lambda

1. Create new lambda function
2. Run time: Python 2.7
3. Select blueprint > Blank Function
4. Configure Triggers > Select "SNS" in the gray empty box on left of lambda logo-sign
5. SNS topic: select the topic you have created above. (SNSForwardEmails)
6. Enable Trigger: check the box
7. Create the function
8. Name: LambdaForwardEmails Runtime: Python 2.7 Code entry type: Edit code inline in the text area, copy-paste the whole file I shared here. Role: Choose an existing role Existing Role: the one you have created above. (SNSEmailForwarder)
9. Next > Create Function

Setup SES:

1. Go to https://console.aws.amazon.com/ses
2. Rule Sets (on the left menu)
3. Create a Receipt Rule
4. Rule Set Name: EmailForwardingRules
5. OPTIONAL: enter your domain name without at sign such as: example.com (Add Receipt)
6. Next Step
7. Add Action: S3
8. S3 Bucket: (create sth) Emails Encrypt Message: (uncheck, I am not sure about the consequences of custom encryption) SNS Topic: SNSEmailForwarder (this is important)
9. Create Rule

It seems ok. Test with a real email. (In the Lambda editor) The test scenario AWS provides cannot simulate an email message totally.

Hi I am stuck with an issue in sending a bounce. Just want to ask if my understanding is correct!
You have extracted SNS messageId here and used it in bounce parameters here.

But as I was reading here that message Id of bounce message is required. Which is provided by SES and obtained by using ses_notification['Message']['mail']['commonHeaders']['messageId']

Btw I am getting below error by using either of the messageId:
Client ErrorFailed to generate a bounce for <3d8f617b-bded-559c-b4bb-371bb2ffdbd5>: <3d8f617b-bded-559c-b4bb-371bb2ffdbd5> does not appear to be a valid original message ID.

@pwnptl It is definitely the case that with the SNS notification version you need to get the id from a different field.

data = event['Records'][0]['Sns']
data = json.loads(data['Message'])
message_id = data['mail']['messageId']

@UnquietCode It could be the issue but data['mail']['messageId'] and ['mail']['commonHeaders']['messageId'] will always have same value.

Hey, this works for me. Thank you so much! However, is there a way to change FROM_ADDRESS to whoever originally sent the email? I tried setting 'FROM_ADDRESS' to 'sender' and that was a fail.

@JustinMarotta13 Are you sending a Bounce or new email ?
Have you followed https://docs.aws.amazon.com/ses/latest/DeveloperGuide/verify-email-addresses.html
Your BounceSender/Source parameter also referred here as 'sender@example.com' must be verified by Amazon SES first.

I'm trying to get sending and receiving working with SES so that I can simply receive any email sent to my domain, but receive that actual email itself, so that I can reply. Amazons implementation worked, but it just forwards an eml file. Your implementation worked for me but I can't see who sent the email. It says it's from "forward@mydomain.com". Which makes sense cause that's what I set the FROM_ADDRESS as. But I would like to make FROM_ADDRESS the original senders name and address. Is this possible?

…

On Sun, Jul 5, 2020 at 12:33 AM pawan patel ***@***.***> wrote: ***@***.**** commented on this gist. ------------------------------ @JustinMarotta13 <https://github.com/JustinMarotta13> Are you sending a Bounce or new email ? Have you followed https://docs.aws.amazon.com/ses/latest/DeveloperGuide/verify-email-addresses.html Your BounceSender/Source parameter also referred here as ' ***@***.***' must be verified by Amazon SES first. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://gist.github.com/356dc60f8f6beb8b5070e891adadab96#gistcomment-3364404>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALZVTN57EGH6YMMSY22TXCLRZ77DDANCNFSM4JLDUXEA> .

Where and how are you storing the eml file? what is the content you are getting there ?
Are you trying to send a new email with someone else's behalf? send_email() api sends a new email. You cannot send a email using from_address which is not verified in your aws account.

So is it possible to take the email and redirect it? The eml file I was talking about is the way Amazon implements forwarding. Their version of your lambda takes the new email from the S3 bucket, and copies it to an eml that is sent via email to an account of your choice. That's just tough to view in gmail.

…

On Sun, Jul 5, 2020 at 1:32 AM pawan patel ***@***.***> wrote: ***@***.**** commented on this gist. ------------------------------ Where are you storing the eml file? Are you trying to send a new email with someone else's behalf? send_email() api sends a new email. You cannot send a email using from_address which is not verified in your aws account. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://gist.github.com/356dc60f8f6beb8b5070e891adadab96#gistcomment-3364431>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALZVTN55K3C2YTNE23VWJ3TR2AF6HANCNFSM4JLDUXEA> .

If you use third party websites to forward the email, the from address will stay the same as the original sender. I was wondering if this is possible through lambda.

…

On Sun, Jul 5, 2020 at 8:43 AM Justin Marotta ***@***.***> wrote: So is it possible to take the email and redirect it? The eml file I was talking about is the way Amazon implements forwarding. Their version of your lambda takes the new email from the S3 bucket, and copies it to an eml that is sent via email to an account of your choice. That's just tough to view in gmail. On Sun, Jul 5, 2020 at 1:32 AM pawan patel ***@***.***> wrote: > ***@***.**** commented on this gist. > ------------------------------ > > Where are you storing the eml file? > Are you trying to send a new email with someone else's behalf? > send_email() api sends a new email. You cannot send a email using > from_address which is not verified in your aws account. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <https://gist.github.com/356dc60f8f6beb8b5070e891adadab96#gistcomment-3364431>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/ALZVTN55K3C2YTNE23VWJ3TR2AF6HANCNFSM4JLDUXEA> > . >

Not sure about SES or Lambda but SNS can forward emails to different email address.

Do you have any recommendations as to implementing that?

…

On Sun, Jul 5, 2020 at 10:26 AM pawan patel ***@***.***> wrote: ***@***.**** commented on this gist. ------------------------------ Not sure about SES or Lambda but SNS can forward emails to different email address using original sender. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://gist.github.com/356dc60f8f6beb8b5070e891adadab96#gistcomment-3364693>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALZVTNZSFWWPAPNELHC7AQTR2CETBANCNFSM4JLDUXEA> .

currently you must be doing SES -> lambda.
you can do SES-> SNS ->lambda. This way you can forward emails in SNS level and do other stuff in lambda. I cannot find a way to forward using lambda and original sender.

One reason it should not exist in lambda level is that lambda can hamper the original email content/fields and sending such content using original sender email address is a security risk and impersonation. Correct me if my hypothesis wrong here.

Makes total sense. Thanks so much for your help!