DeepSeek, a Chinese start-up, has revolutionized the field of artificial intelligence by developing one of the most powerful A.I. systems using significantly fewer computer chips than previously thought possible. Typically, A.I. companies require supercomputers with around 16,000 specialized chips to train their systems, but DeepSeek managed its system with only about 2,000 chips. This breakthrough has significantly reduced the cost of building A.I. technology, with DeepSeek spending only $6 million on computing power compared to Meta’s technology that cost ten times more.
The key to DeepSeek’s success lies in several technological strategies, including the use of a method called “mixture of experts” and a simple mathematical trick involving decimals that optimizes the efficiency of the neural network. By pairing smaller “expert” systems with a “generalist” system, DeepSeek was able to enhance the coordination of interactions between different aspects of the A.I. system. Additionally, the use of fewer decimals for each number in calculations and then stretching the final answer across more bits of memory enabled DeepSeek to create a highly efficient neural network.
While the tricks employed by DeepSeek may seem simple, the complexity of writing the necessary computer code to optimize the GPUs demonstrates the advanced skill level required. Despite the potential for cost reduction and efficiency, many A.I. labs have been hesitant to take the risk and invest in experimental methods due to the substantial funds and potential for failure involved. DeepSeek’s willingness to experiment and share its methods has the potential to significantly impact the cost and advancement of A.I. technologies in the future.
Source
Note: The image is for illustrative purposes only and is not the original image of the presented article.
