Overview

When Beeswax receives an impression notification back from an exchange, that data is first processed by Beeswax and stored in an internal Beeswax database. A job then runs hourly querying Beeswax’s internal data tables to emit batched Win Logs as well as populate aggregated reporting in Report Builder. If a disruption to this process occurs, Beeswax will take corrective action to ensure customer Win Logs reflect complete and accurate impression data.

There are two distinct scenarios in which Beeswax can take corrective action to ensure Batched Win Logs contain correct and accurate data. Each of these has a unique workflow for corrective action.

Delayed Impressions vs Backfills

Delayed Impression Delivery

• When Does this Occur: If there is a disruption or delay in the delivery of impressions to Beeswax’s internal databases that causes impressions to be missed in their expected hourly window of Log emission.
• How Does Beeswax Address: Delayed impressions will be emitted to customer Win Logs as processed during the next scheduled batched delivery. The reported bid_time will remain the time of initial Impression’s occurrence only arriving in a later file delivery. These are delivered within the standard Win Log Folder alongside standard impression delivery, not in the /backfill directory.
  • Delays < 72 hours from Bid Time: Will be automatically processed and delivered in Win Logs, in accordance with Beeswax’s standard impression processing window. 
  • Delays > 72 hours from Bid Time: Beeswax will post a Status Page alert noting the issuance of delayed impressions and the rx_timestamp date range for which the impressions are being issued. 
• Customer Action: No unique action required. Utilize bid_time or rx_timestamp to organize impressions by bid/impression time vs log delivery window.

Backfills

• When Does this Occur: Impressions have been successfully emitted in Win Logs, but Beeswax has since detected that there was a methodology error rendering one or more field’s data incorrect for the emitted impressions. 
• How Does Beeswax Address: Beeswax will reissue corrected impression data and deliver it to the /backfill directory within a customer’s S3 bucket. A Status Page will be posted alerting customers to the backfill and communicating the field(s) that is/are affected, what is changing, and the rx_timestampdate range of impacted impressions. 
• Customer Action: If the impacted field is a field that is important to your data architecture, ingest the impressions from the /backfill directory to replace the corresponding impression data previously delivered, deduping on auction_adgroup_id. If the impacted fields are not relevant to your data architecture, no action is required.

Win Log Deduplication

As with all other Log Types, Beeswax uses auction_adgroup_id as our unique key in Win Logs. Customers should always use auction_adgroup_id for deduplication.

Antenna Win Logs

Win Logs delivered through Antenna are subject to the same delayed impression delay and backfill occurrences as batched Win Logs delivered to S3. For Antenna Win Logs, however, no direct customer action is needed as Beeswax will take the following actions for each respective scenario. The Status Page alerting criteria will be the same as noted above for S3 Win Logs. Neither scenario requires deduplication. 

• Delayed Impression Delivery: Will be automatically processed and delivered in the shared table.
• Backfills: Beeswax will directly rectify the data in the shared table in Antenna. No separate delivery will be made.

Data De-Duplication

Beeswax Data Infrastructure uses an "at least once delivery" design pattern to ensure all events are eventually delivered to customers. In certain scenarios, this may mean that duplicative data is sent in logs to customers. Beeswax always recommends de-duplicating your log-level data on auction_adgroup_id in the case of bid logs or conversion_id in the case of conversion logs.