Programmatic Advertising: The complete Life-Cycle for the beginners
Recently I was having a discussion with my office colleagues regarding the Ad-Tech ecosystem and I realized that most of the newcomers, in fact even the mid-level engineers don’t have clarity on the basic parties involved in a Programmatic Advertising ecosystem. I find similar unawareness among my friends who works in different domains, most of the general audience won’t understand the complexity and different players involved in online programmatic advertising. Upon discussion with them I understood their problem — there are lots of material available on the internet but very few starting from scratch and most of the resources try to explain each play in a silo not the whole life cycle of an online ad getting displayed.
So, in this post, I am trying to explain from scratch different players involved and one simple end to end life cycle of a programmatic RTB Ad-impression. We will not take a deep dive into any of the individual players but will just cover whatever is required to understand the full life cycle. I will be discussing in simple and plain language without any jargon, as we discuss in an informal discussion.
Some terminology before we start:
Programmatic Advertising: Simply put, it is the automated buying and selling of online advertising.
Advertiser: is someone who is trying to sell the products/services, for example, Ford, Coca-Cola, Nike, etc. They pay to get their advertisement displayed on different digital channels.
Publishers: is someone who displays the advertisements, any social media site, blogs, website which has advertisement displayed — Spotify, ESPN, FB, etc. They get paid for displaying ads on their page/app.
Inventory: The actual space on the page where the Ads will be displayed is called inventory. Publisher sells these spaces as an inventory. There are some common types of inventories in a display advertisement as shown below:
Impression: is the term used for Ad getting viewed. When an Ad gets loaded on a webpage that is considered as an impression.
Clicks: When a user clicks on the Ad being showed it’s considered as ‘clicks’.
One Advertiser and One Publisher
Now let’s start. We will go step by step, starting from the most basic scenario — one Advertiser [Sports Shoes — Nike] and one Publisher [Sports Site — ESPN].
ESPN wants to sell its inventories and Nike wants to display its Ads at appropriate places. Nike can go to ESPN directly and ask to display their Ads. As Nike, it makes sense for you to display sports shoes Ads to someone who is visiting sports sites (this is a very simplified version but let’s go with this as of now). As ESPN you will be happy to display NIKE’s Ads on your site as your users will be getting Ads as per their taste (this is again a much-generalized version). More the users visit ESPN sites more the NIKE (advertisers) will be willing to buy from ESPN — Network Effect. Many times, companies like NIKE will hire an Agency to buy the publisher’s inventories on their behalf. These Agencies have media planners who work directly with publishers to buy inventories. These are also referred by some as Agency Trading Desk or just Trading Desk.
Is the above model realistic and feasible?
If there are only these two entities in the Advertising world, then everyone is happy and we need no discussion further but it’s not so, hence we will continue our journey. Now imagine a real-world scenario where we have multiple websites to publish Ads and multiple Ads to be published. In such a case, ESPN will rarely sell its whole inventory to NIKE and Nike will publish all its Ads on ESPN site — this will be disastrous for NIKE because Ads are just a way for Advertiser to convince a user to buy their product. If Nike publishes all its Ads on ESPN, in a way, NIKE is missing all the users who didn’t visit the ESPN site. So what can NIKE do? Visit all the relevant websites and pay to publish their Ads? Unless and until anyone is living in the woods, they will know there are millions of websites if not infinite. Hence, this idea is not feasible. In the same way, ESPN can’t reach all the Advertisers and hence its inventories can go unsold.
Ad-Networks to the Rescue
It seems we need an entity that connects with various publishers and, let’s say in a layman term, collects various publisher’s inventories, and sells them to advertisers. This will solve Advertiser’s problems to a great extent, they don’t have to go to each publisher instead they will contact this entity and this entity can have their Ads published on one of the available inventories (matching the advertiser criteria). Advertising Industry thought the same — we have this entity and it’s called Ad-Networks. Ad-Network generally gets leftover inventories from the publishers, and they are grouped under various categories such as Vertical Ad-Networks, Blind Ad-Networks, and Target Ad-Networks. As we mentioned earlier we will not go into the details of these individual players.
Point to Note: When a new entity is added, it’s not that the old connection is removed completely i.e. Publishers and Advertisers will still have a direct connection for some premium Ads, etc.
** In all the diagrams now onwards, the solid lines show our flow; the dotted lines are showing different possibilities in real-time.
Which Ad-Network to select?
Many people saw this opportunity and hence, many Ad-Network came into the market. Now, for a publisher emerges another problem — which one of these Ad-Network to use? To solve the problem, a new technology platform emerged whose role was to analyze the performance of the Ad-Networks i.e. to find out which one of these will sell a publishers’ inventories at the best price. These Ad-Tech platforms were called Network Optimizers.
The emergence of a new Ad-Tech platform: SSP
In the late 2000s, Real-Time Bidding was introduced in the Programmatic Advertising world, Network Optimizers saw this as an opportunity and they morphed into new Ad-Tech platforms which we know today as Supply Side Platform [SSP]. As we now know, SSPs help in Ad-Network optimization for publishers. So, it makes more sense for an SSP to sit between an Ad-Network (not always but in general) and a Publisher, as shown in the below figure.
A Publisher can mention the floor pricing for a given type of inventories for example if it’s a header at homepage then most likely floor pricing will be more compared to secondary page bottom inventory. We will see later this helps in yield optimization for the Publishers during RTB. A single SSP connects to various Ad-Networks and Advertisers ( for time being, I am using term Advertisers but soon we will see there are different entities with different names and functions), this gives publisher multiple bidders and hence prevents inventory value from falling.
What about Advertisers?
The most obvious thought now will be, SSPs are helping publishers to sell the inventories but what about Advertisers/Media Agencies who are buying the inventories? And the answer is ‘DSP’ — Demand Side Platform.
DSPs are the counterpart of SSPs at the buying side of the ecosystem. DSPs help advertisers/agencies to buy inventories on an impression-by-impression basis from various entities such as SSPs, Ad-Networks, publishers, etc. But DSP’s function is not just simply buying the Ads, usually, an advertiser sets up its campaign in the DSP mentioning creatives (Ads) and various targeting criteria such as age, gender, location, the category of content on site and many more. Each time when an impression is available for the sell, DSP analyses the value of the impression for the advertisers based on the information it has regarding the impression either directly from its user profile database or from third-party data providers known as Data Management Platform (DMP).
DMPs are software platform that collects, stores and organizes the user’s data. They gather data from various sources and data gathered are often categorized into 3 types -
First-Party Data: information gathered directly from users’ interactions e.g. CRMs, Transactional systems, etc.
Second Party Data: These are mainly first-party data from another company. For example, when a user visits online sports shoe site (S1) a cookie is created and S1 can sell this data to Marathon even organizing site (S2).
Third-Party Data: Data collected from any other source is termed as third party data
Point to note: DMPs are not only for DSPs/Advertisers but it can benefit SSPs/Publishers too. By understanding its user better publisher/SSPs can sell the impression/inventories at an optimal price.
Something seems to be missing — where do they buy and sell?
We have covered various kinds of Ad-Tech platforms but something seems to be missing in the ecosystem — isn’t it? Buyers and Seller side seem to be disconnected, to be precise there is no exchange of information between them for buying and selling. If you feel so, you are right we are missing an important component of the market place where the Publishers will put their inventory/impressions for sale and Advertises will buy the same. In the Programmatic RTB advertising ecosystem, we call it Ad-Exchange, similar to a stock exchange.
Seller side sends information related to the impression to be sold such as website, browser, ad-types, user-related info if any, etc. to the Ad-Exchange. Ad-Exchange then broadcasts bid requests to multiple DSPs, Ad-Networks, and Agencies. Each bidder then evaluates the impression and provides its optimal bid price. It sends the optimal bid price and creatives to the Ad-Exchange. Ad-Exchange then selects the winning bid usually using a second-price auction, recently Google and other companies have started moving to the first-price auction.
First Price Auction: The highest bidder wins the auction and makes the payment equal to the bid amount.
Second Price Auction: The highest bidder wins the auction but pays the amount equal to the second-highest bid plus $0.01. Floor pricing, as we saw earlier, comes handy for a publisher to restrict bidding price to fall below the floor price.
Once the winner is selected — the bid price and URL are passed back to the Seller side. Ad-Exchange can be Open or Private. Private Ad-Exchange is used generally for premium Ads and as the name suggests it’s limited to certain publishers and publishers can restrict which advertisers can bid. Publishers prefer this as they can get more price and Advertisers prefer this as they will know where their Ads are getting displayed — brand image/dilution concerns.
So now we are done — right?
It may look like the ecosystem is complete but not so fast. We have, deliberately, left one important piece of the puzzle — the Ad-Servers. When online advertising started to grow publishers soon realized that they need some platform to run and maintain the online Ad campaigns. This led to the birth of Ad-Servers. Today, in addition, to run and maintain the online Ad campaigns, Ad-Servers are responsible for making instantaneous decisions on what Ads to show on a website, then serving them, and collecting and reporting the data such as impressions, clicks, etc. to gain insights.
Ad servers can be used by publishers (known as first-party ad servers) and advertisers (known as third-party ad servers). Both Ad-Servers are essentially the same technology but they are used by publishers and advertisers for different reasons.
First Party Ad-Servers: It allows publishers to manage ad slots on their websites and display ads that have been sold directly to advertisers via direct campaigns. In case no direct Ad-Campaign is found then it goes SSPs, Ad-Networks, and Agencies, etc. to serve the impression. This is also used for targeting and inventory forecasting.
Third-Party Ad-Server: Advertisers use this to track their Ad campaign. Advertisers Ad-Tag data is loaded by Publisher AdServer using which the clicks, impression data is recorded/tracked by the Advertiser Ad-Server.
While the content on the publisher’s website is loading the Ad-space (inventory) has the code which sends out a request to Publisher’s Ad-Server to fill the Ad slot. So, the Ad-Servers are the first direct connection with a Publishers in the Programmatic/RTB advertising ecosystem (simplified one).
Putting all the Pieces Together: The complete Life Cycle
Now, it’s time to put all the pieces together and walk through a full cycle from the point a user visits a publisher’s webpage till the point Ad is loaded. Please refer to the full flow diagram below [excuse the diagram’s dullness — I just prepared it quickly] for the reference. The actual scenario will be too complex hence I have simplified a little bit — the solid lines show our flow; the dotted lines are showing possibilities in real-time.
Let’s take an example — Muskaan: 20–30 years, cricket fan, female — who was browsing dress on the Shein site visits ESPN site. We will use the 200ms lifecycle to show the different events happening at different timestamps (serial number mentioned is the number matching to the diagram).
1. 00ms: Muskaan clicks on the ESPN site.
2. The browser sends a request to the publisher Web-server to load HTML content.
3. The browser starts rendering the content of the page
4. 10ms: Publishers check Data Management Platform [DMP] whether it has information stored for Muskaan or not.
5. 30ms: Publishers sends whatever information it’s able to get, to its AD-SERVER. It checks whether there is any campaign running that has a target customer profile similar to Muskaan. If the match is found then the ad will be served then and there.
6. 40ms: If no campaign targeting Muskaan is found then Publisher Ad-Server then tries to programmatically serve the Ad. Publisher Ad-Server] sends the details to-Traders, Ad-Network, and SSP[Supply Side Platform].
7. 65ms: If the impression is not yet cleared then the server tries to clear the impression by going to Private Exchange.
8. 75ms: If the impression is not cleared from the Private Ad Exchange then Ad-Server sends it to the Open Ad-Exchange [OAE].
9. 80ms: OAE: sends out the bid request with information related to Muskaan, Website, Browser, Ad-Types, etc. to the — Traders, Ad-Networks and DSPs
10/11/12. 100ms: Each bidder goes through the information sent and adds Marketers targeting and budget rules. Each DSP will review the targeting criteria. Let’s say DSP2 has targeting criteria — female, 20–30 age range, location — India, sports fan. DSP1 and DSP3 have slightly different targeting criteria. Based on these, the DSP’s algorithm selects the creative and the optimal bid price — for DSP2 this impression is more valuable compared to other DSPs hence it bids more for it. This bid is then sent back to the Ad-Exchange.
125ms: Ad exchange then selects the winning bid from all the bidders using 2nd price auction.
13. 150ms: Ad-Exchange sends the winning bid and Ad URL to the Publisher Ad-Server.
14. Publisher Ad-Server tells Muskaan’s browser which Ad to display — it sends out the Ad-Tag containing a URL pointing to the Advertiser Ad-Server.
15. 175ms: Muskaan’s browser sends the request to Advertiser Ad-Server and counts the impression.
16. The advertisers Ad-Server sends the markup to the Muskaan’s browser/site and
17. The Ad is displayed.
200ms: Advertiser/winning Bidder Ad-Server receives Ad-Tag data on Muskaan’s initial experience with the Ad.
Please note the whole life-cycle we discussed is a generic simplified version of Programmatic Advertising. Nowadays, almost all the Publishers/Advertisers use a customized solution as per their needs. But I hope this post has brought some clarity to the readers how a Programmatic Advertising works!!
