285: 6 years of cloud news… and we’re still talking about FPGAs and PowerPC

Welcome to episode 285 of the Explain it to me Like I’m 5 Podcast, formerly known as The Cloud Pod – where the forecast is always cloudy! We’ve got a lot of news this week, including the last of our coverage from re:Invent, ChatGTP Pro, FPGA, and even some major staffing turnovers. Titles we almost went with this week: Throw $200 dollars in a fire with ChatGPT Pro Jeff Barr is wrapped up by Agentic AI ️The Tribble with Trilliums ️The Wind in the Quantum Willows ⚰️Rise of the dead instances FPGA and PowerPC Jeff Barr is replaced by Nova The Cloud Pod: Return of the dead instances types After 6 year Jeff Barr hands over the reigns to the CloudPod ⌚For our 6th birthday Jeff barr Retires For our 6th birthday jeff barr delegates announcements to the cloud pod 6 years of meaningless PR drivel ‍6 years of cloud news and we still don’t know what Quantum computing is A big thanks to this week’s sponsor: We’re sponsorless! Want to get your brand, company, or service in front of a very enthusiastic group of cloud news seekers? You’ve come to the right place! Send us an email or hit us up on our slack channel for more info. General News HAPPY 6th BIRTHDAY! 2:00 HashiCorp at re:Invent 2024: Security Lifecycle Management with AWS Hashi is a big sponsor of re:Invent , so of course they had some news of their own to release. HCP Vault Secrets auto-rotation is now generally available. Dynamic secrets are generally available via HCP Vault Secrets. Secrets sync will help keep your secrets synced with AWS Secrets Manager . It still appears to be one direction, but you can now also view secrets in AWS Secrets Manager that are managed by vault. HCP Vault Radar , now in beta, automates the detection and identification of unmanaged secrets in your code, including AWS infrastructure configurations 03:10 Matthew – “ This qualifies under the category of things that I feel like we talked about so long ago, I just already assumed was GA. I’m surprised that it wasn’t.” 03:34 HashiCorp at re:Invent 2024: Infrastructure Lifecycle Management with AWS Terraform AWS provider is now at 3 billion downloads . The AWS Cloud Control Provider is also now generally available with the 1.0 release. This is the provider built around AWS Cloud Control API to bring new services to Hashicorp Terraform faster. In June, AWS and Hashi partnered to co-develop a comprehensive set of terraform policies in compliance with standards like CIS , HIPAA, FINOS and the AWS Well-Architected Framework. In beta now, pre-written sentinel policy sets for AWS available via the Terraform Registry . Supported services include: EC2, KMS, Cloudtrail, S3, IAm, VPC, RDS, EFS Terraform Stacks are now in public beta to simplify infrastructure provisioning and management at scale. When deploying and managing infrastructure at scale, teams usually need to provision the same infrastructure multiple times with different input values, across multiple cloud provider accounts, regions and environments and landing zones. Before stacks, there was no built-in way to provision and manage the lifecycle of these instances as a single unit in Terraform, making it difficult to manage each infrastructure root module individually. 05:43 Ryan – “ I’m a big fan of doing policy evaluation at, you know, Terraform and VolkTime just to get that feedback directly to whoever’s executing that Terraform, rather than have it be a security ticket later or just blocked by permissions. I feel like it’s very good feedback. So having pre-built policies makes life easy, because developing those policies isn’t exactly fun, but that’s super cool.” 08:14 Terraform 1.10 improves handling secrets in state with ephemeral values Terraform 1.10 is now generally available , with several new features, including: Handling secrets. Ephemeral Values to enable secure handling of secrets. Before secrets get persisted in the plan or state file. Since the secrets are stored in plaintext within these artifacts, any mismanaged access to the file would compromise the secrets. To address this, ephemeral values. These values are not stored in any artifact. Not the plan file or the statefile. They are not expected to remain consistent from plan to apply, or from one plan/apply round to the next. Ephemeral supports marking input and output variables as ephemeral. Within ephemeral blocks, which declare that something needs to be created or fetched separately for each terraform phase, then used to configure some other ephemeral object, and then explicitly closed before the end of the phase. 09:22 Ryan – “ I’ve had to battle this with security teams who are looking at, you know, approving Terraform enterprise. I’ve had people pull secrets out of the state file and then use them inappropriately. This is a great feature to see. So pretty psyched about it.” 09:49 Intel CEO Gelsinger forced out after board lost confidence in turnaround plan . Interestingly on stage at AWS, they made the claim that 50% of new CPU capacity was on AWS Graviton. Note great for Intel. CEO Pat Gelsinger has been forced out of Intel after 4 years, handing control to two lieutenants as they search for a successor. Reports are that he left after a board meeting where the directors felt his plan was too costly and ambitious to turn Intel around – efforts so far weren’t working, and the progress of change wasn’t fast enough. Because yeah replacing the top guy is a sure fire way to make things happen faster… Gelsinger inherited a company in 2021 rife with challenges which he only made worse in many aspects. He made claims about AI chip deals that exceeded Intel’s own estimates, leading the company to scrap revenue forecasts a month ago . The full results of his turnaround won’t be known till next year, when he plans to bring a flagship laptop chip back into its own factories. Intel started construction on a $20B suite of new factories in Ohio , and hired a larger workforce to try and reclaim the crown. This eventually led to layoffs and potential sales or spinouts of assets. Gelsinger’s plan included becoming a major player in contract manufacturing for others, a business model called “foundry”. Intel has announced foundry customers including Microsoft and Amazon, but neither would bring to INtel the volumes of chips needed to reach profitability. (I mean at least until it’s proven it works). In addition, they were looking to TSMC to build some of its chips, at the same time trying to compete with TSMC resulting in them not getting great pricing on TSMC fab. 11:37 Jonathan – “ We could do a whole episode on the screwups that Intel’s made over the years. I think they just got, they were in such a dominant position and they became complacent into a risk averse, which is kind of funny to hear that the board were complaining that Gelsinger’s plan was too risky, basically is what they were saying. So they were too risk averse, they still are risk averse. They never took AMD seriously as a competitor…I don’t think anybody could have turned Intel around in 4 years.” AI Is Going Great – Or, How ML Makes All its Money 17:00 Introducing ChatGPT Pro Open AI is adding ChatGPT pro, a $200 monthly plan that enables scaled access to the best of OpenAI’s models and tools. This plan includes unlimited access to their smartest model, OpenAI o1, as well as to o1-mini, GPT-4o and advanced voice. Yeah sorry… I also canceled this subscription recently. 17:37 Jonathan – “ I cancelled my ChatGPT subscription a long time ago and switched to Claude and that’s $20 a month and I regularly run out of credits on there. I would imagine it’s comparatively priced in terms of the number of tokens in and out every day. I mean, I know some people are shocked by the cost, like, my God, it’s $200. But really think about the productivity increase that I’ve seen in using AI over the past few months. I’d pay it in a heartbeat, you know, if Anthropic had an equivalent plan, $200 a month, unlimited access to Claude, even slightly slowed down, you know, I don’t necessarily need like instantaneous responses, but the value you’re getting for $200 is immense.” AWS 20:40 Now Available – Second-Generation FPGA-Powered Amazon EC2 instances (F2) Justin was actually surprised about this announcement – one that they didn’t cover at re:Invent – but that there is a second FPGA powered instance at all. AWS is announcing the F2 instance with up to 8 AMD Field-programmable gate arrays (FPGAs), AMD EPYC (Milan) processors with up to 192 cores, high bandwidth memory, up to 8 TiB of SSD based instance storage and up to 2 TiB of memory, the new F2 instances are available in two sizes, and are ready to accelerate your genomics, multimedia processing, big data, satellite communication, networking, silicon simulation and live video workloads. Some cool examples of how you might use these things: Genomics – Astrazeneca used thousands of F1 instances to build the world’s fastest genomics pipeline, able to process over 400k whole genome samples in under two months. They will adopt Illumina DRAGEN for F2 to realize better performance at lower cost. Satellite operators are moving from inflexible and expensive physical infrastructure (modulators, demodulators, combiners, splitters, etc) toward, agile software-defined, FPGA powered solutions. Using DSP (digital Signal processor) elements on the FPGA, they can be reconfigured in the field to support new waveforms and meet changing requirements. Combined wit the 8 FPGAs, generous amounts of network bandwidth and support for the Data Plan Development Kit and Virtual Ethernet satellite providers can support processing of multiple, complex waveforms in parallel. Neroblade SQL processing Unit (SPU) integrates with Preso, Spark, and other open source query engines, delivering faster query processing and market-leading query throughput efficiency when running on F2. 22:39 Ryan – “ Yeah, I didn’t understand what it did then. I don’t understand what it does now.” 26:37 Introducing storage optimized Amazon EC2 I8g instances powered by AWS Graviton4 processors and 3rd gen AWS Nitro SSDs EC2 I8g instances are now available to you! These new storage optimized instance types provide the highest real-time storage performance among storage-optimized EC2 instances with the third generation of AWS Nitro SSDs and AWS Graviton Processors . AWS Graviton 4 is the most powerful and energy efficient processor they have ever designed for a broad range of workloads running on EC2 instances using a 64-bit ARM instruction set architecture. I8g is the first instance type to use third-generation AWS Nitro SSds. These instances offer up to 22.5 TB of local NVME SSD storage with up to 65% better real-time storage performance per TB and 60 percent lower latency variability compared to the previous generation of i4g. You can get these new shiny instances with up to 96vcpu, 768gb of memory and 22.5 tb of storage. Usual network caps and ebs caps are there with smaller instances, etc. Amazon suggests you consider these servers for I/O intensive workloads that require low-latency access to data such as transactions databases, real-time databases, noSQL and real time analytics such as Spark. 28:02 Matthew- “ I always liked the iSeries. I’ve used them a few times. The free storage there when you don’t care about this type of data and it’s really truly ephemeral or you built it so you have, you know, three NoSQL replicas and you know, one each AZ gives you that free storage layer and doesn’t really cost you that much extra is really nice. And this performance of it was, you know, blazingly fast when I think I did it with the i3. So I can’t imagine what the i8 is.” 29:31 And that’s a wrap! Jeff Barr is announcing that after 20 years, 3283 posts, and 1,577,105 words he is wrapping up as lead blogger on the AWS news blog. Jeff is apparently stepping back to being a builder, and says he went from a developer who could market to a marketer who used to be able to develop. While there is nothing wrong with that, he wants to go back to building. He will still appear on the AWS OnAir twitch show and will speak at community events around the globe, but will be primarily building. But don’t worry – there is a robust AWS News blog team that will keep cranking out new announcements for us to cover. All of us at TCP look forward to seeing what Jeff gets up to next! 30:08 Justin – “ I look forward to seeing what you’re up to next and if there’s a new lead blogger – or if lead blogger becomes Nova over time.” GCP 31:28 New Cassandra to Spanner adapter simplifies Yahoo’s migration journey Cassandra, a key value noSQL database, is prized for its speed and scalability, and used broadly for applications that require rapid data retrieval and storage such as Caching, Session management, and real-time analytics. The simple key value pair structure gives you high performance and easy management, especially for large datasets. But the simplicity means poor support for complex queries, potential data redundancy and difficulty in modeling intricate relationships. To help solve this, they are making it easier than ever to switch from Cassandra to Spanner, with the introduction of the Cassandra to Spanner Proxy Adapter, an open source tool for plug and play migrations of Cassandra workloads to Spanner, without any changes to the application logic. If you’re wondering if the proxy adapter will scale for your needs, don’t worry. Its battle tested by none other than Yahoo. “The Cassandra Adapter has provided a foundation for migrating the Yahoo Contacts workload from Cassandra to Spanner without changing any of our CQL queries. Our migration strategy has more flexibility, and we can focus on other engineering activities while utilizing the scale, redundancy, and support of Spanner without updating the codebase. Spanner is cost-effective for our specific needs, delivering the performance required for a business of our scale. This transition enables us to maintain operational continuity while optimizing cost and performance.” – Patrick JD Newnan, Principal Product Manager, Core Mail and Analytics, Yahoo To get started here are the high level steps to taking advantage of the new Proxy Adapter: Assess your schema, data model and query patterns to determine which you can simplify after moving to Spanner Schema Design. Luckily the table declaration and data types are similar to Cassandras, and with spanner you can take advantage of relational capabilities and features like interleaved tables for optimal performance. Migrate your data through either a bulk load or use Cassandra’s CDC for real time replication. Setup the proxy adapter and update your Cassandra configuration. Test it thoroughly – not in production first Cutover to the new adapter. 32:45 Ryan – “ I didn’t work with the context team much when I was there (Yahoo) but I was on the platform engineering team that sort of created the internal services that provided this functionality. And one of the things that was just starting as I was leaving is the migration to Cassandra away from our internal tool. So it’s exciting. That’s how long ago it was. But it’s, from a Google perspective, that’s a fantastic business model, right? If you can get people using your service by making it really easy to adopt, and then as they slowly transition, you know, the application can probably get better functionality and more features by calling it natively. And it’s a lot easier to consume rather than like a giant migration and rewrite type of thing.” 36:28 Improve your security posture with expanded Custom Org Policy Google is adding support for more than 30 additional services to Custom Org Policies . Originally limited to GKE, DataProc, Compute Engine and Cloud Storage, they are adding some very common ones include BigQuery, Cert Manager, KMS, Load Balancing, NGFW, Cloud Run, Cloud SQL, Cloud VPN, Data Flow, Firewstore, IAM, Identity Platform, Redis, PSC, Secret Manager and VPC. This allows you to enforce conditional restrictions such as specific roles to resources in a project. You can also now set custom org policy to Domain Restricted Sharing principals including all users of an org, specific partner identities, service accounts and service agents. 37:36 Ryan – “ I want to grant everyone primitive roles so I don’t have to manage like very fine grained policies, but I also don’t want them to create, you know, API keys that are going to get proliferated everywhere. And so now with this policy, you can say, you know, you can’t, even with all the permissions, you can’t export this big query dataset to somewhere public or, you know, that, depending on what the conditionals allowed are. So that’s pretty cool. I like that.” 38:29 Introducing Gemini 2.0: our new AI model for the agentic era Google says Sit down Nova… announcing a week after re:Invent the Gemini 2.0 model is available and ready for the agentic era. Of course, this announcement comes just 2 weeks after Justin cancelled his Gemini subscription. Figures. A year ago Gemini 1.0 was launched , with the intent to focus on information as the key to human progress. The first Gemini model built to be natively multi-modal, Gemini 1.0 and 1.5 drove big advances with multi-modality and long context to understand information across text, video, images, audio and code, and process a lot of it. Gemini 2.0 is the most capability multi-modal capable model yet per google. With new advances in multi-modality like native image and audio output and native tool use, it will enable them to build new AI agents that bring them closer to their vision of a universal assistant. Gemini 2.0 flash experimental model will be available to all gemini users. And they are launching a new feature called Deep Research , which uses advanced reasoning and long context capabilities to act as research assistant, exploring complex topics and compiling reports on your behalf. Available in Gemini Advanced today. 2.0 flash replaces 1.5 flash and outperforms 1.5 and even outperforms 1.5 pro on key benchmarks. (See article for some examples) Updates to Project Astra that they announced at I/O. From feedback they have made improvements with the Gemini 2.0 version of Astra. Better dialogue, new tool use including google search, lens and maps. Better memory allowing up to 10 minutes of in -session memory and improved latency. Project Mariner is a new agent that helps you accomplish complex tasks. Starting with your web browser. This research prototype is able to understand and reason across information in your browser screen, including pixels and web elements like text, code, images and forms. Jules is a new AI agent to assist developers with code. It integrates directly into your github workflow. It can tackle an issue, develop a plan and execute it, all under a developers direction and supervision. 40:41 Justin – “ I think it’s just the way to announce a new model. And then they give you some purpose-built agents versus having to build agents from scratch, which you said you would do before.” 41:33 Announcing the general availability of Trillium, our sixth-generation TPU Trilium the 6th generation TPU is now generally available. Trillium TPUs were used to train the new Gemini 2.0 , google’s most capable AI model yet. Some of the key improvements of Trillium: 4x improvement in training performance 3x increase in inference throughput 67% increase in energy efficiency 4.7x increase in peak compute performance per chip Doubled the high bandwidth memory Doubled the interchip interconnect bandwidth 100k trillium chips in a single Jupiter network fabric Up to 2.5x improvement in training performance dollar and up to 1.4x improvement in inference performance dollar. 42:15 Jonathan – “… relative to the old ones. Okay. Yeah, that’s a slight red flag for me. Maybe an orange flag. They’re not comparing it with things that people actually know.” 42:46 Registration is open for Google Cloud Next 2025 Google Next returns to Beautiful Las Vegas at Mandalay Bay, April 9th-11th, 2025. In fact you can register now using the last bits of your 2024 budgets. Early bird pricing is $999 for a limited time (February 14th or when tickets sell out – whichever comes first.) Experience AI in action! Forge Powerful Connections (meet The Cloud Pod Hosts) Build and Learn Live. 43:08 Ryan – “ I’m terrified of what they mean by experience AI in action. Absolutely terrified.” 44:47 ¡Hola Mexico! Google Cloud region in Querétaro now open Google cloud is opening their 41st cloud region in Queretaro, Mexico. This is the third cloud region in Latin America, after Santiago, Chile and Sao Paulo, Brazil. 45:12 Matthew – “ It’s amazing how many regions all these cloud providers have. It used to be like, my god, they’re opening a region. Now it’s like, right, they’re opening another region. Like, is news now, cool.” 45:32 (Re)Introducing IBM Power for Google Cloud Google is reminding you that they continue to offer IBM Power systems on the Google Cloud. Originally launched in 2020, the service then partnered with Converge Technology Solutions in 2022 to upgrade the service by enhancing network connectivity and bringing full support to the IBM i operating system. Today, their announcing Converge Enterprise Cloud with IBM Power for Google Cloud , or simply IP4G supports all three major environments in Power: AIX, IBM i and Linux. It is now available in 4 regions; two in Canada and two in EMEa – in addition to the two in North America. “Infor was one of the original IP4G subscribers, and years later, we continue to run mission-critical IBM Power workloads in IP4G for our clients. IP4G’s availability and performance have more than met our requirements, and we are extremely satisfied with our overall IP4G experience.” – Scott Vassh, Vice President, WMS Development 46:13 Matthew – “ This just feels like you’re not cloud native.” 48:38 Achieve peak SAP S/4HANA performance with Compute Engine X4 machines For those of you trying to make HANA scale GCP has a new machine type for you the X4 . The X4 is purpose-built to handle the demanding workloads of SAP Hana OLTP and OLAP workloads. These machines deliver strong performance, scalability, and reliability empowering businesses to unlock the full potential of their SAP S/4 Hana, SAP Business Suite on SAP HANA and SAP Industry Solutions. X4 is also built to support OLAP workloads such as BW/4HANA and BW on HANA Please no follow up questions on what any of those HANA things are. X4 is available in 16tb, 24tb, and 32tb memory configurations and 960, 1440, 1920 vCPU cores respectively with “standard sizing” SAP certification for SAP HANA OLTP and OLAP capabilities. The X4 16tb machine achieved an SAP benchmark result that was 8% higher than the closest IaaS solution (thanks Google) They are also the only cloud provider providing a certified 32TB SAP machine. “In the past few years, our SAP HANA systems have seen significant data growth with an increasing need for higher performance. With the 24TB X4 machines and Hyperdisk storage, we have been able to raise the ceiling for our future data growth and are also looking to see improvements in our performance. Added to this, Google’s X4 machines are cloud native, giving us opportunities to automate system management and operations.” – Shawn Lund, US Chief Technology Officer, Deloitte 49:08 Justin – “I’ll I’ve learned about SAP HANA is that I don’t ever want to manage it.” 50:42 Introducing Google Agentspace: Bringing AI agents and AI-powered search to enterprises Google is introducing Google Agentspace , which is a *terrible* name. Agentspace unlocks enterprise expertise for employees with agents that bring together Google’s advanced reasoning, Google-quality search and enterprise data regardless of where it is hosted. It will make your employees highly productive by helping them accomplish complex tasks that require planning, research, content generation, and actions all with a single prompt. Agentspace unlocks enterprise expertise by: New ways to interact and engage with your enterprise data using NotebookLM. Including NotebookLM Plus , your employees can upload information to synthesize, uncover insights, and enjoy new ways of engaging with data, such as podcast audio like summaries and more. Information discovery across the enterprise including searching unstructured data such as emails and documents. Expert agents to automate your business functions like expense reports, or other multistep processes. Azure 52:46 Microsoft debuts Phi-4, a new generative AI model, in research preview Microsoft has also told Nova to sit down. Phi-4 is improved in several areas over its predecessor per Microsoft, particularly in math problem solving. Phi-4 is available in limited access via Azure AI Foundry development platform and only for research purposes. This is Microsoft’s smallest model, coming in at 14 billion parameters in size. It competes with other small models such as GPT-4o minim, Gemini 2.0 flash and Claude 3.5 Haiku. Microsoft attributes its performance improvements to high-quality synthetic datasets alongside high-quality datasets of human-generated content and some unspecified post-training improvements. 53:29 Justin – “ They tell you it’s for research purpose only and then it goes and becomes very toxic, you can just say, well, it was only in research.” Oracle 54:57 Oracle Database@AWS Available in Limited Preview For those waiting with baited breath for Oracle Database@AWS, you might still be waiting unless you can get into the limited preview. “ Up until now, it has been impossible to replicate the performance and functionality of Oracle Database on Exadata in AWS ,” said Dave McCarthy, research vice president, IDC. “ With Oracle Database@AWS, customers can finally enjoy that same experience with an easy migration path to the cloud for their on-premises mission-critical workloads. This allows them to reap the benefits of simplifying their daily management and operations to prioritize modernization initiatives. ” “ We want our customers to have access to our data services and to be able to seamlessly use multiple clouds,” said Karan Batta, senior vice president, Oracle Cloud Infrastructure. “This partnership provides a unified way for customers to use the best of Oracle and AWS to take advantage of the latest AI innovations and simplify operations. The introduction of Oracle Exadata Database Service in the AWS US East Region is only the beginning, and we plan to continue to work with AWS to meet customer demand. ” 55:42 Justin – “ I mean, I feel like they’re actually, I think, I think this is exit. think they’re actually installing exit data in the data center. And this hardware is highly tuned for this purpose.” Closing And that is the week in the cloud! Visit our website, the home of the Cloud Pod where you can join our newsletter, slack team, send feedback or ask questions at theCloud Pod.net or tweet at us with hashtag #theCloudPod