📨 Weekly digest: 15 2024 | Is AI infinite?
If yes, what’s the context? | AI this week in the news; use cases; for the techies
Hello friends,
Google just dropped the "Infinite Context" paper: “Efficient Infinite Context Transformers with Infini-attention1.”
A new paper by researchers at Google claims that large language models (LLMs) can work with text of infinite length. The paper introduces Infini-attention, a technique that configures language models in a way that extends their "context window" while keeping memory and compute requirements constantly.
Given how critical long-context LLMs are becoming, having an effective memory system could unlock powerful reasoning, planning, continual adaption, and capabilities not seen before in LLMs.
But if we think about this, AI itself isn't infinite in a couple of ways:
Limited by data and processing power: AI relies on the data it's trained on and the computing power available. While AI can access and process massive amounts of information, it's not limitless. New information or situations outside its training data might confuse an AI.
Focus on specific tasks: AI is typically designed for specific tasks. Even powerful AI systems might not be able to perform well outside their designed area. There's no current AI with general intelligence that can tackle any problem a human can.
However, AI can seem limitless in a few respects:
Continuous learning: AI can keep learning and improving as long as it has new data. This makes its capabilities seem potentially boundless, especially compared to fixed programs.
Rapid innovation: The field of AI is constantly evolving with new approaches and techniques. This ongoing progress can create the impression that AI is approaching limitless intelligence.
We know AI is a powerful tool with vast potential, but it's not infinite. It has limitations and works best within defined areas.
What do you think?
If you haven't already, you can start with our workbook, Building a data-driven organization.
I am looking forward to reading your thoughts in a comment.
Happy days,
Yael et al.
🦾 AI elsewhere on the interweb
Inside Big Tech's underground race to buy AI training data on Reuters
Amazon bets $150 billion on data centers required for AI boom on Bloomberg
U.S. and UK announce partnership on science of AI safety on the U.S. Department of Commerce
Fast access to our weekly posts
🚀 Unbundling AI
📮 Maildrop
🎯 How-tos
🎲 Data and trends
📌 AI case studies
📨 Weekly digest
You are receiving this email because you signed up for Sustainability Insights by Yael Rozencwajg. Thank you for being so interested in our newsletter!
Weekly digest are part of Sustainability Insights, approaches and strategies.
We share tips to help you lead, launch and grow your sustainable enterprise.
Become a premium member, and get our tools to start building your AI based- enterprise.
Not a premium?
Thank you for being a subscriber and for your ongoing support.
If you haven’t already, consider becoming a paying subscriber and joining our growing community.
To support this work for free, consider “liking” this post by tapping the heart icon, sharing it on social media, and/or forwarding it to a friend.
Every little bit helps!