Send Email from VDQ with Anymail

This guide sets up email delivery for the VDQ (VUEDA Dispatch Queue) through Anymail. It covers the backend settings, the provider's tracking webhook, attachments, and how provider events set each Queue Item's state.

VDQ and Background Work Model describes the queue item lifecycle and its known gaps. Set Up and Operate the VUEDA Dispatch Queue (VDQ) covers installing and routing VDQ, creating senders and receivers, running the worker, and retrying, cancelling, and resending items.

Before You Begin

• VDQ is installed, its URLs are included, and a Celery worker is running, as Set Up and Operate the VUEDA Dispatch Queue (VDQ) describes.
• You have an account with an email service provider (ESP) that Anymail supports, and a verified sending domain. Your provider's documentation covers the DNS records (SPF, DKIM) for that domain.

Select the Email Backend

core.default_settings.get_defaults reads the EMAIL_BACKEND config key and puts it in Django's Django: EMAIL_BACKEND setting. The key's default is Django's console backend, which prints each message instead of sending it.

For Mailgun, set these keys in config.local.toml, or as environment variables, which TomlEnv reads first:

EMAIL_BACKEND = "anymail.backends.mailgun.EmailBackend"
ANYMAIL_MAILGUN_API_KEY = "<Mailgun API key>"
ANYMAIL_MAILGUN_SENDER_DOMAIN = "mg.example.com"
ANYMAIL_MAILGUN_WEBHOOK_SIGNING_KEY = "<Mailgun webhook signing key>"
ANYMAIL_WEBHOOK_SECRET = "<random user>:<random password>"

When EMAIL_BACKEND names the Mailgun backend, get_defaults requires the other four keys, and settings fail to load if one is missing. get_defaults also sets ANYMAIL_MAILGUN_API_URL, which defaults to https://api.mailgun.net/v3. Set that key to use another Mailgun API host.

For another ESP, set EMAIL_BACKEND to that ESP's Anymail backend. get_defaults sets no provider settings for it, so add the ESP's ANYMAIL_* settings and ANYMAIL_WEBHOOK_SECRET in your settings module, after the get_defaults line. VUEDA installs django-anymail with the mailgun, sendgrid, and sparkpost extras. Add the extra for any other ESP to your project's dependencies.

Configuration Surface and Defaults describes the use_mailers option, which puts the backend in Django 6.1's MAILERS setting instead.

ESPs without metadata support

By default, VDQ puts each queue item's key in the email's Anymail metadata. A tracking event can then find its item before VDQ stores the message ID. Matching a status update to its queue item describes this fallback. On an ESP that accepts no metadata, Anymail raises AnymailUnsupportedFeature for every email. send_email then logs the exception at info level, builds the email again without the key, and sends it. The ESP receives each email once, and its tracking events match by message ID only.

To skip the second build and the log line, select the no-op strategy in your settings module:

VDQ_EMAIL_TRACKING_STRATEGY = "vueda.vdq.tracking.EmailTrackingStrategy"

This setting changes nothing else about the send. If the ESP can carry the key another way, such as in a tag, write a subclass of EmailTrackingStrategy instead.

With Anymail's IGNORE_UNSUPPORTED_FEATURES setting on, Anymail drops the metadata without raising. The default strategy then sends each email on the first attempt and writes no log line.

Connect the Tracking Webhook

The provider reports delivery results to Anymail's tracking webhook. vdq.urls mounts Anymail's URLs at vueda.vdq/anymail/, below the prefix where you include the VDQ URLs. With VDQ included under routes/, the prefix that the scaffolded project uses, the Mailgun tracking URL is:

https://<user>:<password>@app.example.com/routes/vueda.vdq/anymail/mailgun/tracking/

For another ESP, replace mailgun with its Anymail name, such as sendgrid or sparkpost.

1. Put the two halves of ANYMAIL_WEBHOOK_SECRET in the URL as the user and password. Anymail returns 400 for a webhook request whose HTTP basic auth credentials do not match.
2. Register the URL in the provider's dashboard for delivery, bounce, and failure events.
3. For Mailgun, copy the webhook signing key from the same dashboard into ANYMAIL_MAILGUN_WEBHOOK_SIGNING_KEY. Anymail checks each event's signature with it.

Send an Email

Call vdq.schedulers.add_email with a Sender and one or more Receiver rows:

from vueda.vdq.schedulers import add_email

queue_items = add_email(
    sender=support,
    to=[customer],
    subject="Your order has shipped",
    text="Your order is on its way.",
    html="<p>Your order is on its way.</p>",
    reply_to=[orders_desk],
    origin=order,
)

add_email returns the queue items it created:

• It creates one queue item for each receiver in to, cc, and bcc. Each item sends one message, addressed to its own receiver only.
• The message carries a Reply-To header with the email addresses of the reply_to receivers.
• The sender, every receiver, and every reply_to receiver must have an email address. Otherwise add_email raises ValueError, and its transaction rolls back every row it created.
• origin links each queue item to the record that caused the email. A model that subclasses QueueItemOrigin can reach its queue items through a generic relation.
• add_email publishes the send task when the surrounding transaction commits.

Receivers in cc and bcc each get their own message with no Cc header, so they do not see each other's addresses.

To create the queue items without sending them yet, call vdq.schedulers.add_abstract_email with the same arguments. Call vdq.schedulers.schedule_queue_item on each returned item when it is ready.

Attach Files

Pass attachments as a mapping from file name to a mapping with these keys:

• mimetype: required, such as application/pdf.
• file: required, a file object opened in binary mode. add_email reads it during the call.
• content_disposition_is_inline and content_id_string: optional. Set both to embed an image in the HTML body.

with open("logo.png", "rb") as logo, open("invoice.pdf", "rb") as invoice:
    add_email(
        sender=support,
        to=[customer],
        subject="Your invoice",
        text="Your invoice is attached.",
        html='<p><img src="cid:logo"> Your invoice is attached.</p>',
        attachments={
            "logo.png": {
                "mimetype": "image/png",
                "file": logo,
                "content_disposition_is_inline": True,
                "content_id_string": "logo",
            },
            "invoice.pdf": {"mimetype": "application/pdf", "file": invoice},
        },
    )

vdq.handlers.send_email attaches an inline image under a content ID that Anymail generates. It rewrites each cid:<content_id_string> reference in the HTML to that ID.

add_email saves each file once, under email_queue_item_attachments/ in the default storage (Django: STORAGES). All queue items from the call share those attachment rows. To reuse attachments from an earlier call, pass a list of saved AnyMailQueueItemAttachment rows instead of a mapping.

Serve attachment links

The queue item API lists each attachment with a download URL built from the VDQ_URL setting. get_defaults does not set it. Set it in your settings module to the path of the VDQ URLs:

VDQ_URL = "/routes/vueda.vdq"

Links then point at the vueda.vdq_attachments_retrieve endpoint. Any signed-in user can download an attachment by its ID. VDQ and Background Work Model describes this gap.

Delete attachment files after sending

To delete attachment files once they are sent, set VDQ_MAX_FILES_AGE_IN_SECONDS = 0 in your settings module. When a queue item reaches a Done State, VDQ deletes the file of each attachment whose queue items are all done. The attachment row stays. With any other value, or with the setting unset, VDQ keeps the files.

Resending an item whose attachment files were deleted fails, because the send reads each file. The new item moves to errored.

Provider Errors and Retries

send_email sends through Anymail and updates the queue item from the result:

Result	Queue item
Accepted with Anymail status sent or queued	Stores the provider's message ID in message_id and moves to awaiting.
Accepted with any other status	Records the status in result and moves to errored.
Provider API error with HTTP status 5xx, 408, 423, or 429	Raises AnymailTransientError, and the task retries. The item waits in delayed.
Any other error	Appends the traceback to result and moves to errored.

A tracking event can reach the item before send_email records the result. If the event has already moved the item out of sending, send_email stores the message ID and leaves the state and result unchanged.

The task class BaseTask sets the retry schedule. It allows up to 12 retries (Celery: Task.max_retries) with exponential backoff from 16 seconds (Celery: Task.retry_backoff), without jitter. When the retries run out, the item moves to errored.

Provider Events and Queue Item States

Anymail sends each tracking event to vdq.handlers.handle_bounce. It finds the queue item by the event's message ID. When no item stores that ID, it asks the email tracking strategy for the item. The default strategy reads the queue item key from the event's metadata. Matching a status update to its queue item describes this fallback. The event then changes the item:

Event type	Effect
delivered	Clears result and moves the item to succeeded.
bounced, rejected	Records the provider's reject reason in result and moves the item to errored.
failed	Records Email Failed. in result and moves the item to errored.
deferred	Writes a note to result that the provider will retry. Moves a sending item to awaiting.
queued, sent	Moves a sending item to awaiting. No change for an item in any other state.
Any other type	Logged at info level. No change.

handle_bounce logs an event that finds no queue item as a warning and skips it. VDQ and Background Work Model describes what happens when an event arrives for an item that is in neither sending nor awaiting.

Verify the Setup

1. Send a request to the tracking URL without credentials:

   curl -i -X POST https://app.example.com/routes/vueda.vdq/anymail/mailgun/tracking/

   The response is 400, because the basic auth credentials are missing.

2. Send a test event from the provider's dashboard. The server log shows Tracking event <type> for unknown message_id=<id>, because the test event does not name a queue item.

3. Send an email with add_email to an address you control.

4. Sign in as a user whose groups grant vueda_vdq.read_queueitem, and fetch the item by its ID from the queue item endpoint (vueda.vdq_queueitem_list). Its workflow_state_code is awaiting after the send, and succeeded after the provider reports delivery.