Sakana Creates More Sustainable Nature Inspired AI Models

The generative AI landscape is growing rapidly, with AI models like GPT and LaMDA and large language models like Bard and Claude dominating the market. Like every other industry, the AI market is also filling up with smaller companies and startups now developing their own models.

With an increased supply of AI models and a higher level of production, a massive amount of data and computing power comes along, whether for a model’s scalability or for improved performance. This amount of data and power comes with its perks, but it has become not only costly for AI model developers but also far from sustainable, negatively impacting the environment in numerous ways.

Several ways to mitigate the high cost of building AI models have been found. However, Sakana AI, a startup founded by ex-Google researchers Llion Jones and David Ha, explores the root of the problem by embracing collective intelligence* and biomimicry* in the development of its nature inspired AI model.

*Collective Intelligence: The process by which a large group of individuals gather and share their knowledge, data, and skills for the purpose of solving societal issues.

*Biomimicry: A practice that learns from and mimics the strategies found in nature to solve human design challenges.

Solutions to Data and Power

For companies that exist and continue to thrive in the tech industry, like OpenAI or Google, not much has stopped them from building AI models. Many have even managed to enlarge their models by scaling up transformer models*, for instance, though the process is still far from economical.

*Transformer Model: A neutral network used to learn, process, and understand context and thus meaning by tracking relationships in sequential data.

However, the high cost of building these models has become a major barrier for smaller businesses and organizations that wish to integrate AI into their workflow. Due to this, more and more ways of reducing the cost of AI development, as well as its unsustainable effects, are being implemented.

This includes the use of cloud computing platforms like AWS or Azure, which offer services used to train and deploy models at a lower cost compared to on-premises infrastructure. A number of these services, along with research labs from big tech companies, produce pre-trained models, which smaller companies or organizations can utilize to eliminate the costly need to train a model from scratch. Plenty of companies have also leveraged open-source tools and frameworks like TensorFlow to avoid purchasing propriety software.

There’s no denying that these methods have accommodated the many needs of companies developing AI models and are working quite effectively, as we have witnessed a colossal growth in AI model production. According to the AI Index Report of 2023, over 200,000 AI models are available today, growing quite significantly from around 150,000 models only three months ago.

Despite the conveniences these solutions have provided, they all involve an external force, and Sakana AI aims to “fix” the issue of high costs and unsustainable practices from the very conception of their nature inspired models.

Swarm Approach

Sakana AI harnesses swarm collective intelligence and evolutionary computing principles through biomimicry, in its AI model production.

Having previously worked at Google and are thus aware of Google’s one-type vision of generative AI technology, shown through LaMDA and PaLM, the duo finds current AI model frameworks as restrictive. With a rising number of AI models now following the same formula, Jones and Ha view the existing, dominant commercial trend of AI as less-than-innovative.

Due to this, instead of building a large model that requires much computational power and scaled-up transformer models, as we know many companies have done, the startup envisions a swarm of smaller models that can deliver results as advanced and as complex as bigger existing models do when they come together.

Though the startup hasn’t provided much detail regarding the nature inspired models themselves, the two founders believe that a swarm-based approach can not only provide results equal to larger models but also be more economical, secure, and adaptable.

The duo finds adaptability as one of the strongest factors to consider when building an AI model, like ants dynamically forming a bridge that may not be the strongest but has the advantage of being built right away whenever needed, and can “adapt to the environments.”

Japan Headquarters

Sakana’s name is derived from the Japanese word さかな (sa-ka-na), meaning fish, for the startup to bring forth the idea of a school of fish filled with individual units that can come together and form a coherent entity from simple rules.

Unlike most AI companies that are based in Silicon Valley, the Sakana duo chose Tokyo as the startup’s headquarters, due to the city’s advanced technical infrastructure.

Sakana hopes to attract international tech-savvy individuals with diverse perspectives and carry on with its research and development activities in its soon-to-be-built research lab.

Meme & AI-Generated Picture

Job Posting

• Huntress - Senior Product Designer - US/Canada (Remote)

• Machina Labs, Inc. - Software Engineering Manager - Los Angeles, CA (Hybrid)

• Peapod Digital Labs - User Experience Research Strategist - Chicago, IL+ (Remote/Hybrid)

• Rocket Travel - Senior Software Engineer, Backend - Chicago, IL+ (Remote)

Promote your product/service to Digger Insights’ Community

Advertise with Digger Insights. Digger Insights’ Miners are professionals and business owners with diverse Industry backgrounds who are looking for interesting and helpful tools, products, services, jobs, events, apps, and books. Email us [email protected]

Your feedback would be greatly appreciated. Send it to [email protected]