Open Source Large Language Models and Meta

TLDR: Meta’s decision to open-source Llama 3.1 is a calculated bet on the future of content creation and ad campaign creation. By making AI models like Llama widely available, Meta is poised to benefit from the increased demand for curation, personalization, and distribution of AI-generated content. As the largest social media platform in the world, Meta is well-positioned to capitalize on this trend, and its open-source approach may ultimately prove to be a savvy move that disrupts the business models of its present and would-be competitors.

I’ve been putting Meta’s Large Language Model (LLM) Llama 3.1-405B¹ through its paces lately, and I’ve got to say I’m impressed. There’s been a noticeable increase in my coding velocity on a personal project since committing myself to using Llama over the traditional Google search-and-sift-through-links for programming related queries. What I find most impressive about Llama isn’t its capabilities — after all, I’ve had free access to Gemini Advanced (Google’s best publicly available LLM offering) at work, and it’s similarly effective within a professional software engineering context. Rather, what sets Llama apart to me is its being open-source, and the fact that I can tap into its largest model, with 405 billion parameters, for free, courtesy of meta.ai.

Naturally, this got me thinking: what’s in it for Meta? The costs of training, serving, and running inference on these models are presumably substantial. And yet, they’re giving it away for free, no strings attached – not even a waitlist. All I need is an Instagram login, and I’m immediately good to go peppering Llama with questions regarding web sockets, NATS, and broker architecture design. There has to be a larger strategy at play here . . . valued at $1.29T as of today, Meta is hardly a charity, so what gives? What’s the endgame? My thinking is two-parted, predicated on: (1) how LLMs lower the barrier to content creation, and (2) how LLMs might facilitate ad campaign creation. I’ll tackle these related notions in turn.

Multimodal generative AI has the potential to unleash a torrent of content that’s not only plentiful and varied, but also of surprisingly high quality. I use “quality” advisedly here, acknowledging that it’s a subjective term; at its worst, AI-generated content is vapid and cringeworthy, but at its best, it may be thought-provoking and genuinely impressive. In a world where AI-generated ‘slop’ and gems coexist (a world we arguably already exist in), a pecuniary premium emerges on content’s curation, personalization, and distribution. So, who stands to gain from this shift? One clear beneficiary is Meta; their crown jewels - Instagram and Facebook - thrive based on the engagingness of user-posted content and their ability to monetize said engagement through ads.

Reading (→) as “leads to” or “causes”, the causal chain for (1) is straightforward: a lower barrier to content creation → a surge in diverse content across and adapted for various formats, including text generation (e.g., marketing copy, taglines), audio, and video (script writing, captioning, multilingual translation) → an increased need for curation (filtering and organizing for specific audiences), personalization (audience relevance), and contextualization (stitching content together to make compelling narratives) → a heightened emphasis on strategic distribution, particularly on social media platforms for consumer packaged goods; targeted advertising; and ensures that quality recommendation systems remain a meaningful differentiator. I’d be remiss to not mention that through this chain, compute and accured technical competency is a transferable advantage here, applicable in non-ML or non-LLM use-cases.

It’s in the above context that Meta’s move to build open source models like Llama 3.1 most makes sense to me. Stratechery² puts it well:

“Meta isn’t selling its capabilities; rather, it sells a canvas for users to put whatever content they desire, and to consume the content created by other users. It follows, then, that Meta ought to be fairly agnostic about how and where that content is created; by extension, if Meta were to open source its content creation models, the most obvious place where the content of those models would be published is on Meta platforms. To put it another way, Meta’s entire business is predicated on content being a commodity; making creation into a commodity as well simply provides more grist for the mill.”

It appears that Mark Zuckerberg is making a calculated bet: that the ultimate consumption of content - regardless of whether it’s AI-generated or not –- will occur within Meta’s properties, all of which are primed for monetization. A close reading of his letter justifying the open-sourcing of Llama³, ‘Why Open Source AI is Good for Meta’, suggests as much. While he waxes philosophical about the virtues of open source, I’m not entirely convinced of his sincerity; I detect a hint of restrained fury, a lingering resentment towards Apple and Google, the former especially being the archetypal closed-software bully that has burned him (through Meta) in the past⁴.

Now that we’ve examined the content side from the vantage point of Meta’s users, let’s turn our attention to ad creation, something supremely relevant to Meta’s paying customers, its advertisers. There’s reason to believe that reducing friction in ad creation -– via multimodal gen-AI –- will lead to an uptick in ad campaigns. I wouldn’t be surprised if a non-trivial segment of advertisers are currently constrained by their own creative limitations and prioritization decisions, which, when taken together, relegate ad campaigns to a lower priority. This inertia can lead to a reluctance to launch ad campaigns altogether, or at the very least, a reduced willingness to engage in ongoing monitoring and A/B experimentation of existing campaigns –- a lost opportunity for ad platforms to generate additional revenue; resulting in a lower-than-desired ‘consumption’ of ad campaigns. Gen-AI changes this dynamic in several ways, enabling seamless multilingual translations, allowing advertisers to reach a broader audience without the need for costly and time-consuming translation services. Additionally, gen-AI makes cross-platform ad adaptation easier, automatically adjusting ad formats and creative assets to fit the unique requirements of each platform (ad placement too). This is particularly attractive to SMBs (small businesses and individuals), who might lack the resources and expertise to manage and monitor complex ad campaigns across multiple languages and platforms. By simplifying the ad creation process and unlocking new markets, gen-AI can help SMBs reach a wider audience and drive more sales – a compelling proposition for advertisers who may have historically been deterred by the complexity and cost of ad creation. If the process were made incredibly easy -– requiring only a budget commitment and some basic parameters -– they might be persuaded to take the plunge and launch an ad campaign. With LLMs, Zuck and Meta are a step closer to total ad vertical integration and a world where they can gobble up the entirety of a business’ marketing dollars. (Meta Advantage+ telegraphs this with its “single-step automation” for campaigns across “audience, budget, placement, ad creative and conversion destination”).

Given our earlier suggestion that reducing friction in ad creation could lead to increased consumption of ad campaigns and potentially more bidding over specific audiences or keywords (because of an increase in demand), it’s again somewhat puzzling why an open-source model of Llama would benefit the company. Why not keep it closed-source, integrate it further into their own products, and exclude others from doing the same by not providing any model for free (as they risk divulging their “secret sauce”)? By open-sourcing Llama, Meta appears to be giving away a valuable competitive advantage, allowing other companies to integrate the model into their own ad platforms and potentially siphon off some of Meta’s ad revenue. It seems counterintuitive that Meta would willingly surrender this opportunity for exclusivity and revenue growth. Here too, the (reasonable) assumption that most terminal/ultimate consumption happens within Meta’s properties applies. Just as content is inevitably posted and published in either Facebook, Instagram, or sent over Whatsapp, ad campaigns might inevitably and finally occur through Meta’s ad network (or Google’s, as the ad duopoly leaders). What’s it to Meta if you designed your ad campaign or made your viral picture using Llama, or Gemini, or ChatGPT — so long as you publish through Meta, which you’re wont to given their size of distribution, everybody wins, Meta and its advertisers most of all. Zuck again says as much in his letter:

“Selling access to AI models isn’t our business model. That means openly releasing Llama doesn’t undercut our revenue, sustainability, or ability to invest in research like it does for closed providers”.

Another savvy move because you can undercut competitors who do -— Amazon, Google and Microsoft all have huge cloud services that provide AI models/services to enterprises, ostensibly with a vested interest in serving their expensive models but are forced by the market to offer open source models. By open-sourcing Llama, Meta can disrupt the business model of these competitors, who rely on recouping their investment in AI research through cloud services.

AI models nowadays give me the impression of a commodity. Perhaps their commodification will be permanent. Unsurprisingly, there is significant value in distribution and so the focus remains on building products that attract and retain users, as well as assembling the pieces needed to wield a dominant and durable distribution network. Meta appears well poised to take advantage of this. Good move Zuck, good move.

Notes