Overview

When the platform receives an impression notification back from an exchange, that data is first processed and stored in an internal database. A job then runs hourly querying the internal data tables to emit batched Win Logs as well as populate aggregated reporting in Report Builder. If a disruption to this process occurs, the platform takes corrective action to ensure customer Win Logs reflect complete and accurate impression data.

There are two distinct scenarios in which FreeWheel can take corrective action to ensure Batched Win Logs contain correct and accurate data. Each of these has a unique workflow for corrective action.

Delayed Impressions vs Backfills

Delayed Impression Delivery

• When Does this Occur: If there is a disruption or delay in the delivery of impressions to the internal databases that causes impressions to be missed in their expected hourly window of Log emission.
  
  
• How it is Addressed: Delayed impressions will be emitted to customer Win Logs as processed during the next scheduled batched delivery. The reported bid_time will remain the time of initial Impression’s occurrence only arriving in a later file delivery. These are delivered within the standard Win Log Folder alongside standard impression delivery, not in the /backfill directory.
  • Delays < 72 hours from Bid Time: Will be automatically processed and delivered in Win Logs, in accordance with the standard impression processing window. 
  • Delays > 72 hours from Bid Time: The platform will post a Status Page alert noting the issuance of delayed impressions and the rx_timestamp date range for which the impressions are being issued. 
    
    
• Customer Action: No unique action required. Utilize bid_time or rx_timestamp to organize impressions by bid/impression time vs log delivery window.

Backfills

• When Does this Occur: Impressions have been successfully emitted in Win Logs, but the platform has since detected that there was a methodology error rendering one or more field’s data incorrect for the emitted impressions. 
  
  
• How it is Addressed: The platform will reissue corrected impression data and deliver it to the /backfill directory within a customer’s S3 bucket. A Status Page will be posted alerting customers to the backfill and communicating the field(s) that is/are affected, what is changing, and the rx_timestampdate range of impacted impressions. 
  
  
• Customer Action: If the impacted field is a field that is important to your data architecture, ingest the impressions from the /backfill directory to replace the corresponding impression data previously delivered, deduping on auction_adgroup_id. If the impacted fields are not relevant to your data architecture, no action is required.

Data De-Duplication

The platform's Data Infrastructure uses an "at least once delivery" design pattern to ensure all events are eventually delivered to customers. In certain scenarios, this may mean that duplicative data is sent in logs to customers. FreeWheel always recommends de-duplicating your log-level data on auction_adgroup_id in the case of bid logs or conversion_id in the case of conversion logs.

Win Log De-Duplication

As with all other Log Types, the platform uses auction_adgroup_id as the unique key in Win Logs. Customers should always use auction_adgroup_id for deduplication.

Antenna Win Logs

Win Logs delivered through Antenna are subject to the same delayed impression delay and backfill occurrences as batched Win Logs delivered to S3. For Antenna Win Logs, however, no direct customer action is needed as FreeWheel will take the following actions for each respective scenario. The Status Page alerting criteria will be the same as noted above for S3 Win Logs. Neither scenario requires deduplication. 

• Delayed Impression Delivery: Will be automatically processed and delivered in the shared table.
  
  
• Backfills: The platform will directly rectify the data in the shared table in Antenna. No separate delivery will be made.