Overview

When the platform receives an impression notification back from an exchange, that data is first processed and stored in an internal database. A job then runs hourly, querying the internal data tables to emit batched Win Logs and populate aggregated reporting in Report Builder. If a disruption to this process occurs, the platform takes corrective action to ensure customer Win Logs reflect complete and accurate impression data.

There are two distinct scenarios in which FreeWheel can take corrective action to ensure Batched Win Logs contain correct and accurate data. Each of these has a unique workflow for corrective action.

Delayed Impressions vs Backfills

Delayed Impression Delivery

When Does this Occur

If there is a disruption or delay in the delivery of impressions to the internal databases that causes impressions to be missed in their expected hourly window of Log emission.

How is it Addressed

Delayed impressions will be emitted to customer Win Logs as processed during the next scheduled batched delivery. The reported bid_time will remain the time of initial impression’s occurrence, only arriving in a later file delivery. These are delivered within the standard Win Log Folder alongside standard impression delivery, not in the /backfill directory.

• Delays < 72 hours from Bid Time: Will be automatically processed and delivered in Win Logs, in accordance with the standard impression processing window.
• Delays > 72 hours from Bid Time: The platform will post a Status Page alert noting the issuance of delayed impressions and the rx_timestamp date range for which the impressions are being issued.

Customer Action

 No unique action required. Utilize bid_time or rx_timestamp to organize impressions by bid or impression time vs log delivery window.

Backfills

When Does this Occur

Impressions have been successfully emitted in Win Logs, but the platform has since detected that there was a methodology error rendering one or more field’s data incorrect for the emitted impressions. 

How is it Addressed

The platform will reissue corrected impression data and deliver it to the /backfill directory within a customer’s S3 bucket. A Status Page will be posted alerting customers to the backfill and communicating the affected fields, what is changing, and the rx_timestamp date range of impacted impressions. 

Customer Action

If the impacted field is a field that is important to your data architecture, ingest the impressions from the /backfill directory to replace the corresponding impression data previously delivered, deduping on auction_adgroup_id. If the impacted fields are not relevant to your data architecture, no action is required.

Data De-Duplication

The platform's Data Infrastructure uses an "at least once delivery" design pattern to ensure all events are eventually delivered to customers. In certain scenarios, this may mean that duplicative data is sent in logs to customers. FreeWheel always recommends de-duplicating your log-level data on auction_adgroup_id for bid logs, or conversion_id for conversion logs.

Win Log De-Duplication

As with all other Log Types, the platform uses auction_adgroup_id as the unique key in Win Logs. Customers should always use auction_adgroup_id for de-duplication.

Antenna Win Logs

Win Logs delivered through Antenna are subject to the same delayed impression delay and backfill occurrences as batched Win Logs delivered to S3. For Antenna Win Logs, however, no direct customer action is needed as FreeWheel will take the following actions for each scenario:

• Delayed Impression Deliveries will be automatically processed and delivered in the shared table.
• Backfills will cause the platform will directly rectify the data in the shared table in Antenna. No separate delivery will be made.

The Status Page alerting criteria will be the same as noted above for S3 Win Logs. Neither scenario requires de-duplication.