|
|
Examples of various Beeswax log types can be found on the Beeswax GitHub repository.
The mapping between ID and Inventory Source names is available here.
Detailed definitions of log headers, field lists, and data dictionaries can be accessed in the publicly available Beeswax Log File Header Definitions.
Beeswax log files are provided in CSV format. For users who need JSON format, a CSV↔JSON converter can be used as Beeswax does not support JSON for batch logs.
All Beeswax logs include timestamps in both Eastern Time (ET) and Coordinated Universal Time (UTC). If the timestamp is not labeled with "_utc", it is in ET. Logs may be delayed slightly, and directories for hourly log dumps will reflect the UTC hour.
For Auction and Bid logs, a separate S3 bucket must be provided for each region (e.g., us-east-1, us-west-2, etc.). For Win and Conversion logs, a single S3 bucket can be used across regions.
Beeswax waits 24 hours from the time of the auction to receive impression notifications. After this period, any impression notifications for that auction will be ignored. This typically affects less than 1% of total impressions.
Customizing the folder structure for different log types is possible. Discuss any folder restructuring with your Beeswax account manager for further details.
In cases where the user has not been assigned a user ID or has not accepted third-party cookies, the user ID will be missing from the log entries.
Auction Logs contain all bid requests made to the Bidder instance, capturing key details of the auction process.
Auction Logs are primarily used for analytics, helping users refine their bidding strategies. However, they cannot be used to create derivative segments using the user IDs included in the bid requests.
Auction Logs will only contain up to 600 characters for segment IDs, limited to the requests where a bid was returned. For a complete listing of all segment IDs for a given auction, users should reference Segment Logs and join them on the Auction ID.
Bid Logs are delivered in near-real-time as small CSV files, typically containing a few thousand bids per file. These logs provide insight into bid performance.
Bid Logs are helpful for calculating win rates across various dimensions, such as device type, geolocation, or creative. This data is valuable for optimizing bidding strategies, particularly for users who employ custom bidding agents.
Like Bid Logs, Conversion Logs are delivered in near-real-time as CSV files, containing a few thousand records per file.
Beeswax sends all conversions regardless of attribution status. Conversion Logs include both attributed and non-attributed conversions, providing a comprehensive view of campaign performance.
Conversion Logs enable users to analyze conversions at any time. Since these logs are unjoined, they need to be combined with Win Logs to gain a complete understanding of conversions and related campaign activity.
Batch Win Logs are delivered either hourly or daily. These logs are joined, meaning conversion and click data are matched to impression data.
Daily batch Win Logs are typically written to customers' S3 buckets between 6–7 AM ET (6 AM during daylight savings time, 7 AM otherwise).
Auction IDs may appear duplicated in Batch Win Logs when events are re-rendered from the browser or app. These events will only be de-duplicated at the batch level.
Beeswax automatically de-duplicates impressions in Batch Win Logs using the auction_adgroup_id field.
Streaming Win Logs are available in real-time via HTTP or Kinesis. These logs can be delivered in either JSON or Protocol Buffers (Protobuf) formats, with the option for joined or unjoined logs.
Unjoined logs contain only one type of event per message (Impression, Click, Activity, or Conversion). In contrast, joined logs include a combination of events (e.g., RequestLogMessage + ImpressionLogMessage + ClickLogMessage).
As more data about a win event is received, Beeswax may send multiple messages with the same Auction ID, each containing updated information (e.g., adding click or activity data to the original impression data).
Messages are posted to the stream indefinitely. However, if a message is received more than six hours after the auction, it will not include the RequestLogMessage information. This occurs in less than 1% of cases.
Segment Logs provide a complete list of matched first-party segments, which is essential for audience forecasting and attribution. These logs can be joined with Auction Logs on auction_id to extend segment visibility and improve targeting insights.
Beeswax logs are designed for "at least once delivery," which can sometimes result in duplicate data entries. To ensure accuracy, logs should be de-duplicated using appropriate keys such as auction_adgroup_id (for bid logs) or conversion_id (for conversion logs).
Because Beeswax's data infrastructure does not track historical records, duplication may occur. It is recommended to de-duplicate logs at the auction_adgroup_id level for bid logs and at the conversion_id level for conversion logs.
Bid Logs should be de-duplicated on the auction_adgroup_id field, as multiple bids can be placed for the same auction by different line items.