How is Python Being Used At Facebook?

Python is becoming a very popular programming language and many industries are embracing and developing new applications using this language. Google, YouTube, Netflix, and many others are the core examples of popular companies who are using Python.

But do you know that Facebook, the social media king is using Python for various developmental purposes?

Use of Python at Facebook

Not all the coding of Facebook is written in Python, but still, millions of lines of code are written in Python.

Learning about the fact that production engineers at Facebook commonly use Python programming language may surprise Python expert.

Python is the third most favored programming language that is currently being used at Facebook after PHP and C++.

Usually, production engineers use Python to maintain thousands of libraries and binaries Facebook’s infrastructure.

Read also : How To Make A Website Like Facebook?

In order to achieve the purpose of hardware imaging, infrastructure management, a binary distribution, and operational automation, a number of Production engineers write code in Python services and frameworks.

• Over 21% percent of overall codebase infrastructure is covered by Python at Facebook while rest is covered by Java, C, and C++ with 9%, 15%, and 55% respectively.

• Maintaining thousands of libraries and binaries distribution is done with Python.

• Use of Python’s framework Py3 increased by 5% as of May 2016.

• From 2016 to till date, an average of 5000 codes is modified by more than 1000 committers on

Python in Production Engineering at Facebook

Facebook’s infrastructure team extensively uses Python which can be easily found everywhere in production engineering.

The main focus of the infrastructure team is to deliver and maintain Python client libraries for the interface and binary communication protocol that enables other teams who want to interact with them by offering reliable interfaces.

With the help of these interfaces, the amount of code that is written, tested and maintained by production engineers is reduced, enabling them to move ahead at a faster pace in integrating services to Facebook’s infrastructure.

It also helps to scale up the reliability and efficiency of the entire infrastructure. Python helps in maintaining and managing the infrastructure of Facebook.

Facebook’s production engineer own various software, written in Python that enables them to manage the infrastructure and covers the life cycle of their hardware.

Services Driven by Python at Facebook

Currently, Python is responsible to drive various services at Facebook which are listed below:

• Network Switch Setup and Imaging: Python helps the service to form a successful communication bridge among network devices.

• Whitebox Switch CLIs: Python enables full-scale network deployment that enables the client to manage switches from a CLI interface.

• Service Turn-up: Python provides core service such as DNS, Chef etc turn-up with the support of a pluggable system known as Kobald.

• Hardware Faults: It also enables auto-remediation of service failures and faults in service in hardware.

• Scheduling Maintenance Work: Using Dapper it also helps in automating and scheduling maintenance work.

• Testing and Repair management: Using Cyborg Python helps in burn-in testing, server imaging, and repair management

• Server Check: Also, it helps in detecting fault and diagnosis with the help of Machinechecker. Machinechecker is CLI utility software that checks the health of Facebook’s servers.

Platform Services

Scaling up of the Facebook’s infrastructure has also scaled up the services. Services which were uniform once are now broken into different components.

This division has given rise to a wide variety of general-purpose services for Python.

Read also : How To Make a Social Media Website?

The infrastructure management tools that are used by production engineering are built on various platforms. Hence, these are listed below:

• Job Engine: A framework which is used to scale up job scheduling and execution that can be extended by any team.

• fbpkg: It is a blob distribution service that allows the transfer of large files and software packages including binary of facebook.com.

• FBTFTP: It is an open source framework to build high-performance TFTP servers. Currently, they use high-performance IPv6 TFTP.

• Osmosis: It is an execution tool used by teams for office or data center development and kernel and OS updates.

Service Configuration Management at Facebook

Facebook’s uses Chef (automated IT infrastructure tool) to manage their host-level configuration while to manage their service-level configuration they use Configerator (an authored project of Facebook).

Production engineers use Python to write code which is executed to produce configuration objects.

These objects are later stored in JSON format that can be easily consumed by any other service.

Also, Validators are used to ensure whether these configuration objects are defined correctly or not which is also written in Python.

Python can also be used as a configuration language for container deployment system for Facebook, Tupperware.

Improvement in Operational Efficiency

The building on top of existing libraries and system has helped many teams in improving their general or own operations at Facebook.

Using Python, MYSQL infrastructure team has developed a service known as MySQL Pool Scanner that keeps their database up to date.

This tool automatically manages and maintain databases in the way same as a database administrator.

Another tool which they have built on top of job engine using Python is Orchestrator that allows their widely distributed binaries to roll out safely with safety checks.

Deployment of Python 3 in Facebook Codebase

Currently, Facebook codebase uses Python 2 and Python 3 for their various libraries and models (such as Gevent or Twisted).

All the projects on Facebook mainly use Python 3 unless there is a need of using Python 2.

In production engineering, over 5% of Python services are running in Python 3. FBOSS CLI, Facebook Python Ads API, PYAIB, FBTFTP are the Python 3 compatible projects.

Facebook uses Real-Time Framework for Python

 It has become important for a social network side to have real-time updates that make it simple for the people to share with friends.

News feed needs to be updated in real time to make stream relevant and engaging.

To deal with this issue, Facebook acquired Friend Feed’s real-time web framework known as Tornado that helped them to build a site which is updated in real-time.

Tornado is a Python web framework originally developed by the FriendFeed team to handle thousands of connections simultaneously.

Thus, it becomes easier for Facebook to enable real-time services. Initially, it was designed to handle a large amount of web traffic with speed.

Read also : Python Structure For Your Project: How to do it?

Tornado is a similar Python web framework to Django, Google’s web app or web.py that also provides other features like security, localization, user authentication or cookie handling.

With the help of the Python web framework, it has become possible for Facebook to ensure that user’s news feed is updating regularly in real-time.