Be part of our every day and weekly newsletters for the newest updates and unique content material on industry-leading AI protection. Study Extra
Liquid AI, a startup co-founded by former researchers from the Massachusetts Institute of Know-how (MIT)’s Pc Science and Synthetic Intelligence Laboratory (CSAIL), has introduced the debut of its first multimodal AI fashions.
Not like most others of the present generative AI wave, these fashions will not be based mostly across the transformer structure outlined within the seminal 2017 paper “Attention Is All You Need.”
As a substitute, Liquid states that its aim “is to explore ways to build foundation models beyond Generative Pre-trained Transformers (GPTs)” and with the brand new LFMs, particularly constructing from “first principles…the same way engineers built engines, cars, and airplanes.”
It appears they’ve carried out simply that — as the brand new LFM fashions already boast superior efficiency to different transformer-based ones of comparable dimension equivalent to Meta’s Llama 3.1-8B and Microsoft’s Phi-3.5 3.8B.
Often called the “Liquid Foundation Models (LFMs),” these fashions at present are available three completely different sizes and variants:
- LFM 1.3B (smallest)
- LFM 3B
- LFM 40B MoE (largest, a “Mixture-of-Experts” mannequin much like Mistral’s Mixtral)
The “B” of their title stands for billion and refers the variety of parameters — or settings — that govern the mannequin’s data processing, evaluation, and output era. Usually, fashions with the next variety of parameters are extra succesful throughout a wider vary of duties.
Already, Liquid AI says the LFM 1.3B model outperforms Meta’s new Llama 3.2-1.2B and Microsoft’s Phi-1.5 on many main third-party benchmarks together with the favored Large Multitask Language Understanding (MMLU) consisting of 57 issues throughout science, tech, engineering and math (STEM) fields, “the first time a non-GPT architecture significantly outperforms transformer-based models.”
All three are designed to supply state-of-the-art efficiency whereas optimizing for reminiscence effectivity, with Liquid’s LFM-3B requiring solely 16 GB of reminiscence in comparison with the greater than 48 GB required by Meta’s Llama-3.2-3B mannequin (proven within the chart above).
Maxime Labonne, Head of Publish-Coaching at Liquid AI, took to his account on X to say the LFMs had been “the proudest release of my career :)” and to make clear that the core benefit of LFMs: their skill to outperform transformer-based fashions whereas utilizing considerably much less reminiscence.
The fashions are engineered to be aggressive not solely on uncooked efficiency benchmarks but in addition by way of operational effectivity, making them superb for a wide range of use instances, from enterprise-level functions particularly within the fields of monetary companies, biotechnology, and shopper electronics, to deployment on edge units.
Nonetheless, importantly for potential customers and clients, the fashions will not be open supply. As a substitute, customers might want to entry them by means of Liquid’s inference playground, Lambda Chat, or Perplexity AI.
How Liquid goes ‘beyond’ the generative pre-trained transformer (GPT)
On this case, Liquid says it used a mix of “computational units deeply rooted in the theory of dynamical systems, signal processing, and numerical linear algebra,” and that the result’s “general-purpose AI models that can be used to model any kind of sequential data, including video, audio, text, time series, and signals” to coach its new LFMs.
Final yr, VentureBeat lined extra about Liquid’s strategy to coaching post-transformer AI fashions, noting on the time that it was utilizing Liquid Neural Networks (LNNs), an structure developer at CSAIL that seeks to make the substitute “neurons” or nodes for remodeling knowledge, extra environment friendly and adaptable.
Not like conventional deep studying fashions, which require hundreds of neurons to carry out complicated duties, LNNs demonstrated that fewer neurons—mixed with modern mathematical formulations—might obtain the identical outcomes.
Liquid AI’s new fashions retain the core advantages of this adaptability, permitting for real-time changes throughout inference with out the computational overhead related to conventional fashions, dealing with as much as 1 million tokens effectively, whereas preserving reminiscence utilization to a minimal.
A chart from the Liquid weblog reveals that the LFM-3B mannequin, as an illustration, outperforms in style fashions like Google’s Gemma-2, Microsoft’s Phi-3, and Meta’s Llama-3.2 by way of inference reminiscence footprint, particularly as token size scales.
Whereas different fashions expertise a pointy enhance in reminiscence utilization for long-context processing, LFM-3B maintains a considerably smaller footprint, making it extremely appropriate for functions requiring massive volumes of sequential knowledge processing, equivalent to doc evaluation or chatbots.
Liquid AI has constructed its basis fashions to be versatile throughout a number of knowledge modalities, together with audio, video, and textual content.
With this multimodal functionality, Liquid goals to deal with a variety of industry-specific challenges, from monetary companies to biotechnology and shopper electronics.
Accepting invites for launch occasion and eyeing future enhancements
Liquid AI says it’s is optimizing its fashions for deployment on {hardware} from NVIDIA, AMD, Apple, Qualcomm, and Cerebras.
Whereas the fashions are nonetheless within the preview part, Liquid AI invitations early adopters and builders to check the fashions and supply suggestions.
Labonne famous that whereas issues are “not perfect,” the suggestions obtained throughout this part will assist the group refine their choices in preparation for a full launch occasion on October 23, 2024, at MIT’s Kresge Auditorium in Cambridge, MA. The corporate is accepting RSVPs for attendees of that occasion in-person right here.
As a part of its dedication to transparency and scientific progress, Liquid says it can launch a sequence of technical weblog posts main as much as the product launch occasion.
The corporate additionally plans to have interaction in red-teaming efforts, encouraging customers to check the bounds of their fashions to enhance future iterations.
With the introduction of Liquid Basis Fashions, Liquid AI is positioning itself as a key participant within the basis mannequin area. By combining state-of-the-art efficiency with unprecedented reminiscence effectivity, LFMs supply a compelling different to conventional transformer-based fashions.