X (formerly known as Twitter) remains one of the most recognizable social media platforms in the world, with millions of daily active users. The network’s impressive scale has meant that X has run into several bottlenecks since its launch in 2006. It’s also experienced significant changes in the past few years after its well-publicized change in ownership to Elon Musk at the end of 2022.

We’ve detailed a full history of the Twitter tech stack in this article, including which solutions the platform relied on at launch and how these have evolved in the preceding couple of decades. We’ll also outline how its algorithm works and how X’s new tech stack differs from previous iterations under Musk’s new direction.

The Early Days of the Twitter Tech Stack

Twitter was founded in 2006, and the first version of its social media platform was launched that year. This first version of Twitter was built using the Ruby programming language, specifically through the server-side web app framework Ruby on Rails.

Ruby on Rails, sometimes abbreviated to Rails, represented a reasonable choice for Twitter in its early stages. Rails had only launched a couple of years prior in 2004 and represented an innovative framework with many new features that supported rapid development. This innovation led to many similar success stories, with other large scale apps like Airbnb, Twitch and Shopify all being built using Rails frameworks.

They also relied on third-party hosting at this time, and all of its data was stored on MySQL. Initially, they relied on a small database, which later evolved into a larger single database and then several clusters of large databases as the platform and its users grew. All of these initial technology stacks served their purpose until around 2009 when Twitter began many of its more substantial migrations.

How the Twitter Tech Stack Evolved: Migrating from Ruby to Java

The most significant shift in how Twitter’s technology stack operated was its shift away from Ruby on Rails. Twitter’s popularity exploded around 2009; it achieved large scale 1,444% growth between June 2008 and 2009, with the average time spent on the site also up by 175%. This meant many more user interactions happening across the platform, with massive amounts of tweets being sent every day.

As the number of users and the amount of tweets being sent grew, Twitter’s Ruby on Rails framework began to buckle under the pressure. Twitter had designed a humorous message for anyone not able to access the platform, showing a whale being lifted up by several of its logo doves. This “fail whale” would come to define the platform for the next few years as it appeared so frequently whilst the platform worked on Rails. 

In 2009, they began the complex process of switching from Ruby to JavaScript. This switch was finally completed in April 2011, when the company announced they were now operating from a Java server called Blender. The shift was even more successful than any of the company’s developers had planned; Twitter initially expected a 10x improvement in performance, but the more scalable systems they created were actually able to go from around 200-300 requests per second per host to closer to 10,000-20,000. They also started using Apache Mesos around this time, in 2010.

Twitter would eventually retire the fail whale image that it had historically relied on for years completely in 2015 in an attempt to shrug off the lingering impacts of the image. These early days of Twitter and its necessary migration are a great example of why companies need to tackle their technical debt before they get out of hand.

Twitter’s (X’s) Focus on Scalability

After moving from Ruby to JavaScript, Twitter worked to further transition towards a complex technology stack that could react and scale alongside its user base. They also shifted the Twitter backend to Scala, operating through the Java Virtual Machine (JVM) to function more efficiently than they had using Ruby. They continue to operate from a microservices architecture comprised of several third-party applications and programming languages.

Twitter also found itself processing huge amounts of real-time data due to the frequency of tweets. To support this, they made the critical decision to start using Apache Storm. Apache Storm is the computation framework which allows them to handle millions of tweets per second with real time processing. The Twitter tech stack has also adopted Python as an additional language to support data pipelines and to further support its growing machine learning models. Twitter then also began applying different methods to scale its data centers as well as its search functionality.

How Does Twitter Achieve Scalability Through Data?

Data storage and scaling have been some of the key challenges facing X, as they are common with platforms that host millions of users. As of 2017, they had implemented several data storage solutions to handle the large volumes of tweets being sent by users daily from around the world. 

This included Manhattan, a distributed database they built in 2014 and which began to phase out a previous Apache Cassandra layer of storage. They also relied on Hadoop clusters, which were used primarily for data analytics. To further these data access efforts, in 2022, they addressed their cloud services system by migrating to Google Cloud. In the same year, they also began experimenting with new data quality automation solutions.

How Does Twitter Achieve Scalability Through its Search Tools?

Twitter has also needed to scale its search functionality to support users across the platform. Twitter engineers came up with an Elasticsearch solution in 2022, which is a popular open-source tool. They added a reverse proxy to Elasticsearch, which could separate read and write traffic, which helped them create better metrics for searches.

They also applied this thinking to a common problem on Twitter; traffic spikes. The platform had been known to experience huge spikes in traffic during certain breaking news days or events. They created an ingestion service to stop these spikes from overloading search clusters, and also introduced a backfill service to further improve the platform’s search functions.

X Tech Stack: How Has Twitter’s Tech Changed Since Elon Musk Took Over?

The Twitter technology stack, now known as the X tech stack, has experienced several changes in the past few years since the platform was bought by Elon Musk at the end of 2022. The first of these significant changes involves X’s move towards more algorithmic content through its “For You” feature.

The For You page is designed to deliver tweets to each individual user based on their content preferences. To do this, X operates a three-stage process of candidate sourcing, ranking, and then filtering. 

Candidate sources are first drawn from various sources, including in-network sources from people users actually follow, then out-of-network sources, and then through a procedure of embedding spaces. X’s ranking system then scores every tweet based on tweet interactions. After this, they’re filtered based on a wide range of variables, including content balance, trending topics, social proof and other factors. X uses Apache Thrift to rank tweets. 

Tweets are then mixed and delivered across each user’s For You page. More information on how this algorithm works can be found on the X engineering blog, formerly known as the Twitter engineering blog.

Further to these algorithmic changes, X has also integrated more AI features. Most notable of these is GrokAI, which launched in 2023 and was built using the Grok large language model (LLM). It was built for X in collaboration with xAI, a separate AI and machine learning business founded in March 2023. Grok’s functionalities have expanded in a couple of years since its launch, most recently with Grok 3 launching in February 2025. Grok now represents one of X’s new key components within its user interface.

This is it: The world’s smartest AI, Grok 3, now available for free (until our servers melt).

Try Grok 3 now: https://t.co/Tj0afLoxEz

X Premium+ and SuperGrok users will have increased access to Grok 3, in addition to early access to advanced features like Voice Mode pic.twitter.com/YgKavSCiWr

— xAI (@xai) February 20, 2025

The new direction for X hasn’t meant that they’ve abandoned their scaling efforts. X has also worked on the performance of its Hadoop clusters by optimizing them using Kerberos. These newly kerberized clusters have become a significant cost-saving for the company. 

What Can We Learn from the X Tech Stack?

The early days of Twitter’s migrations show how important it is for a platform to react and respond when it can no longer scale alongside its users, even if this means making critical decisions like adopting entirely different programming languages. At present, X has continued to scale and adapt its performance capabilities with a variety of new solutions and AI integrations, and it remains to be seen how these will support the platform in the future.