DeepSeek releases new open-source Fire-Flyer File System for parallel file system management

DeepSeek AI is continuing its open-source momentum following the release of the DeepSeek-R1 LLM and the image-generating Janus Pro, now unveiling the Fire-Flyer File System (3FS), designed for AI-HPC operations.

Launched as part of the company's Open Source Week initiative, 3FS boasts an impressive 7.3 TB/s aggregate read throughput within its server data clusters. However, this is not a completely new development, as DeepSeek has been utilizing it internally since 2019.

Optimized for rapid random read speeds, 3FS is a Linux-based parallel file system that forgoes traditional read caching to better suit AI models, where vast amounts of training data are processed in single-use, non-repetitive intervals.

An August research paper from last year highlighted 3FS’s role in enhancing the Fire-Flyer 2 deep learning cluster, enabling it to achieve 80% of the performance of NVIDIA’s DGX-A100 server solutions but at only 50% of the cost and 60% of the power consumption. It recorded a 6.6 TB/s throughput, with an additional 1.4 TB/s allocated for background training tasks—significantly outperforming a control set using the Ceph file system, which managed just 1.1 TB/s.

Now publicly available for download and evaluation at no cost, 3FS is expected to draw industrial interest despite its Chinese origins.