Real Time Bidding (RTB) is here to stay. Not surprisingly, it is touted as the revolution that is transforming the very way online advertising works. Demand Side Platforms (DSPs) have a lot at stake, especially so, when in midst of a bidding cycle. DSPs can now leverage  DataTorrent RTS to produce accurate and valuable bids for Real Time Bidding.

Real-time Bidding (RTB)1 is defined as

an auction model for buying display media, whereby advertisers bid on an impression by impression basis and pay only what each impression is worth to them—but at a massive scale. It can lead to increase in ROI  for advertisers, more sales for publishers, and more relevant ads for consumers.”

Demand-side platform (DSP)2 is defined as

“a system that allows buyers of digital advertising inventory to manage multiple ad exchange and data exchange accounts through single interface. Real-time bidding for displaying online advertising takes place within the ad exchanges, and by utilizing a DSP, marketers can manage their bids for the ads and the pricing for the data that they are layering on to target their audiences. DSPs allows users to optimize based on set key performance indicators such as effective Cost per Click (eCPC), and effective Cost per Action (eCPA).”

How does the RTB workflow look like?

This illustration and the accompanying task-by-task steps will show you how the RTB flow works.

  1. A user visits a web page. The ad tag on the web page triggers the ad serving process.
  2. An impression is announced. Exchange “asks” for the bid by sending ad requests to DSPs.
  3. DSPs receive ad requests.
  4. DSPs evaluate the requests and makes the decision to bid or to not bid.
  5. DSPs submit the ad along with a bid price.
  6. Exchange runs the bidding process, and the winner is determined.
  7. Ad is displayed.



As illustrated, for each ad served through RTB, the process needs to be completed under strict timelines. The timeline is critical  because this process executes while the page is loading, and has a direct impact on user experience. Typically, the entire process must complete in 100 ms. Considering the time required for the auction and network latency, a DSP has at most 50 ms to submit a bid. 50 ms for the most valuable decision!


dspBatch Bidding

The bid price for an ad depends on multiple factors.

Typically, a bidding engine needs to access data from different data stores while determining the bid price. Because latency is involved while accessing datastores, the decision-making approach involves the bidder making a bid decision as a batch job. For each ad, the bidder calculates the bid to be submitted with the ad for the next time interval, and stores it in a datastore that the ad server can access. The ad server selects the best ad for the current cycle, and sends it back to RTB exchange with the bid price that was computed by the bidding engine for that ad.Clearly, there are several drawbacks to this approach:

  • Bid price is  based on older data.
  • Bid price is used for a longer interval, typically in the order of several minutes.
  • New data points can be introduced, but this will  involve accessing more databases, thereby affecting the time for calculating bids.
  • Scalability is a concern.

To obviate these drawbacks we need:

  • The ability  to access the most recent data.
  • Faster feedback for recent bid decisions, thus enabling the analysis of the impact of such decisions
  • The possibility of taking subsequent corrective measures.
  • The ease of integrating data from other sources without affecting latency, thereby allowing developers to focus solely on improving their algorithms.

How does DataTorrent RTS play a role in enhancing this flow?


Datatorrent RTS, industry’s only big data platform that unifies stream and batch processes, can effectively provide enhanced value to the RTB flow.

  • DataTorrent RTS can process massive amounts of streaming events
    A bidder can receives billions of auction requests per day and peak loads could go up to 300-400K requests per second or more. DataTorrent RTS can deftly process large volumes of data in real-time, thus allowing bidders to intelligently handle incoming auction requests.  
  • The performance and scalability becomes a matter of complexity as data volumes increase in size
    In an RTB flow, data volumes are only on the increase, with more publishers and ad exchanges in market. DataTorrent RTS is built to scale, thus functioning as a platform that not only handles existing large volumes, but smoothly accommodates sudden spikes in data flow.
  • Rapid integration with existing infrastructure and data sources
    Data can fall in different categories:  ad requests, tracking logs, user profiles, historical data, and campaign goals. This data can reside in a variety of data stores, thus implying that an integration of the big data analysis platform with the existing infrastructure is a must! Such a platform should be able to process this data, and emit useful output, such as alerts, dashboards, and automated reactions. DataTorrent RTS holds an obvious advantage in this scenario; it comes with a unique capability to integrate with different data sources coupled with an out-of-the-box support for generating meaningful output in the form of dashboards. The in-built support minimizes time-to-market for an RTB-specific application development in an environment of changing landscape and requirements.
  • Fast processing leading to quick insights
    A bid value decided using most recent patterns  and  data would definitely prove to be more valuable. The sooner  the most-recent data becomes available, the more accurate would be  thebid price calculation. DataTorrent RTS can work with in-memory data to derive intelligent insights into rapidly flowing data.

With so many competitors bidding for their ad to be served, advertisers betting on DSPs for the success of their campaign, while considering the  impact of bids on the ROI of advertisers, bidding quickly and accurately is critical.  DataTorrent RTS assists at simplifying the RTB flow. DSPs must consider leveraging Datatorrent RTS platform to design their bidding engine solution. Upcoming blog would dive in into more design related details . Watch this space for more!




Download DataTorrent Sandbox here

Download DataTorrent Enterprise Edition here

Join Apache Apex Meetups here