About this episode
Welcome to episode 282 of The Cloud Pod, where the forecast is always cloudy! This week Justin, Ryan, and Matthew are happy to be joining you in the clouds versus watching election information. This week we’re talking nuclear energy, AI Search tools, and all things Pre:Invent. Welcome, and thanks for joining us! Titles we almost went with this week: ️The Cloud Pod Would Much Rather Record This Show Than Watch the Election Results ️IBM Comes for Your AI Dollars AWS Goes Limitless with the PostgreSQL Possibilities ⌚It is Upon Us the Pre-Invent Period and AWS Does Not Disappoint ⚛️Amazon Loses Its Nuclear Superhero A big thanks to this week’s sponsor : We’re sponsorless! Want to get your brand, company, or service in front of a very enthusiastic group of cloud news seekers? You’ve come to the right place! Send us an email or hit us up on our slack channel for more info. Follow Up 01:13 Energy regulators scrutinizing data center use reject Amazon bid Late Friday, the Federal Energy Regulatory Commission rejected a proposal that would have allowed an Amazon data center to co-locate with an existing nuclear power plant in Pennsylvania. The commission voted it down 2-1 FERC chairman Willie Phillips said that the commission should encourage the development of data centers and semiconductor manufacturing as national security and economic development priorities. Commissioners Mark Christie and Lindsay See (both R) voted to reject the proposal, while Davis Rosner and Judy Change (D) didn’t vote. Talen Energy, who signed the agreement, drew challenges from neighboring utilities AEP and Exelon – who challenged the novel arrangement, arguing it would unfairly shift costs of running the broader grid to other consumers. FERC’s order found the region’s grid operator, PJM Interconnection, failed to show why the proposal was necessary and prove such a deal would be limited to the Susquehanna plant given the widespread interest in placing data centers next to power plants. Talen said the ruling would have a chilling effect on the region’s economic development and it is weighing its options. Will see what happens with Microsoft/Constellation energies plan to restart 3-Mile Island. 3:21 Justin – “It’s sort of sad because I kind like the idea of nuclear power to solve a bunch of problems, but it has to be done in the right way for sure.” General News 04:12 IT’S EARNINGS TIME! 04:22 IBM revenue misses, but execs say AI will drive future growth This week, we have an additional company we don’t typically talk about… but IBM kicked off this quarter’s earning seasons which indicated that the AI dividend has yet to pay off for big infrastructure players. Earnings were 2.30 a share, excluding non-recurring items, and were eight cents better than consensus estimates. Although revenue rose 2% on a constant currency basis to $14.97 billion, they were slightly below the $15.08 billion consensus. “We are very focused on ensuring we get an early lead position and establish IBM consulting as a strategic provider of choice for gen AI,” said Chief Financial Officer James Kavanaugh. “This is a long-term growth factor with a multiplier effect across our software, our platforms and our infrastructure.” About three-quarters of the gen AI business is consulting, and one-quarter is software. 05:32 Ryan – “…it seems like that’s a pretty good play to beef up, you know, your consultant side of the business to implement that. Because a lot of businesses are going to need to do that. And a lot of them don’t have the in house skills to do it.” 05:50 Alphabet stock soars as earnings crush estimates on strong cloud growth Alphabet reported earnings per share of $2.12 on revenue of $88.27 billion for the quarter ended September 30th. Representing a profit and sales increase from the same period last year of 37% and 15%, respectively. Analysts had expected revenue of $1.83 per share and 86.44 billion. Advertising revenue topped expectations at 65.85 billion vs expectations of 65.5. Cloud revenue was $11.4 b up 35% from the same period last year exceeding expectations. Sundar Pichai said “ this business has real momentum, and the overall opportunity is increasing as customers embrace gen AI ” Google plans to spend $13 billion on capital expenditures. 07:09 Matthew – “I mean, I was talking recently with some people and they were saying how a lot more of the really small companies are leveraging Google just because their developer experience inside the platform is much better than the other ones. It’s interesting to kind of see if that’s it, but it’s a ton of small companies to keep up with.” 07:40 Amazon stock jumps 6%; Q3 revenue up 11% to $158.9B; profits hit $15.3B; AWS sales up 19% Amazon topped estimates for the 3rd quarter, reporting $158.9 billion in revenue, up 11% YOY and earnings per share of $1.43. Profits jumped to 15.3 billion, from 9.9 billion a year ago. AWS came in just below expectations at 27.4 billion in revenue, up 19%, with 10.4 billion in operating income. Investors continue to keep a close eye on AI adoption on the cloud giant. This is all interesting despite layoffs and unfavorable RTO policies; they are currently at 1.55 million employees, up 3% YOY. 09:17 Justin – “…it’s a crazy amount of people, by the way. I can’t even fathom having 1.5 million employees. Like, what do they all do?” 09:39 Microsoft dips on weak guidance after beating on earnings Microsoft reported an earnings and revenue beat for the fiscal first quarter, but was bludgeoned for predicting slower growth than analysts expected. Revenue was 65.59 or 3.30 per share vs the 64.51 or 3.10 per share expected. CEO Satya Nadella said he feels pretty good that going into the second half of this fiscal year that the supply-demand will match up. Azure growth was 33%, with 12 points of that growth coming from AI services. 10:47 Justin – “No, they were, they were applauded for doing well and beating expectations, but they were beaten because they predicted slower growth for this quarter and the next quarter. So it was more, I don’t think they lowered their guidance, but I think they basically said to expect it to be on the lower side of the range that they gave, which made investors unhappy.” AI is Going Great – Or How ML Makes All Its Money 11:38 Introducing ChatGPT search ChatGPT has launched a Chrome extension to take over the search experience from Google. With ChatGPT Search , you can search the web with fast, timely answers with links to relevant web sources, which you would have previously needed to go to a search engine for. ChatGPT will choose to search the web based on what you asked, or you can manually choose to search by clicking the web search icon. Search will be available at chatgpt.com and their desktop and mobile apps. Open AI says that getting answers on the web can take a lot of effort, and sometimes requires multiple searches and digging through links to find quality sources and the right information. Now with chat you can get a better answer. For real time sources, Chat GPT has partnered with news and data providers to get things like weather, stock, sports, news and maps. 12:58 Matthew – “I just like how their first real solution was, hey, let’s do a Chrome plugin, which is owned by Google. You’re just trying a weird next step.” AWS 15:16 Amazon Virtual Private Cloud launches new security group sharing features AWS is making it easier to manage your security groups with a new security group sharing feature. You can associate a security group with multiple-VPCs in the same account using Security Group VPC associations. When using shared VPC, you can now also share security groups with participant accounts in that shared VPC using shared security groups. This ensures security group consistency and simplifies configuration and maintenance for your admins. Now make it possible to publish a managed security group for SaaS services customers may want to connect too…. 16:02 Matthew – “They had something that I definitely used in the past, which was a Lambda that watched the Amazon SNS topic for the public IP addresses. you could block it. In theory, you could do the same thing. Well, especially because you was over the default 50 group limit, 50 rule limit. So every time you wanted to use it, you always had to request the limit upgrade.” 18:08 AWS enhances the Lambda application building experience with VS Code IDE and AWS Toolkit AWS Lambda is giving you a new experience to simplify the development of lambda based apps using VS Code IDE and the AWS Toolkit. This experience streamlines the code-test-deploy-debug cycle, providing a guided walkthrough that assists developers from setting up their local development environment to run their first application on the cloud and adds enhanced user experience in each step in the cycle. When you install the AWS toolkit extension on VSCode, you will be greeted with a new app building experience. It will guide you through the necessary tooling installations and configurations required to set up your local environment for building Lambda-based apps. In addition, you get a curated list of sample application walkthroughs, which guide them step-by-step through coding, testing and deploying their apps in the cloud. 16:02 Ryan – “My first thought when reading this is I’m curious on how this will like sort of fit in with my AWS SAM workflows, which does give you a CI-CD workflow because publishing directly with cloud formation. So it is sort of an interesting thing. I’m hoping that you could kind of seamlessly merge those experiences because it would be kind of nice if they made that easier.” 19:382 AWS Lambda now supports AWS Fault Injection Service (FIS) actions AWS Lambda now supports the AWS Fault Injection Service (FIS) actions. With FIS actions for AWS Lambda, developers and operators can now verify their application’s response to Lambda errors for all language runtimes without modifying the code. Some of the tests can be to return custom HTTP status codes from the gateway or add one second of startup delay to 1% of invocations. It’s nice to have some fault injection opportunities for your Lambda functions at once as well. 12:32 AWS now accepts partial card payments In something that I feel took way too long, AWS customers can now pay with their cards to make partial payments toward their monthly bill. Until now, customers could only pay their entire bill at once, prior to the due date. With partial payments, customers can now split the amount due into smaller payments which can be charged on different cards. To do this previously you would have had to call AWS customer service , but now you can do it from your Console account. 22:12 AWS announces Amazon Redshift integration with Amazon Bedrock for generative AI Announcing general availability of auto-copy for Amazon Redshift Amazon Redshift now supports incremental refresh on Materialized Views (MVs) for data lake tables Announcing Amazon Redshift Serverless with AI-driven scaling and Optimization AWS announces CSV result format support for Amazon Redshift Data API Several new features for Redshift this week including: Redshift Integration with Amazon Bedrock allowing you to leverage large language models from simple SQL commands alongside your Redshift data The next gen AI driven scaling and optimization in cloud data warehousing. Redshift Serverless uses AI techniques to automatically scale with workload changes across all key dimensions such as data volume changes, concurrent users and query complexity. The redshift Data API now supports comma separated values (CSV) result format which provides flexibility in how you access and process data, allowing you to choose between JSON and CSV formats 22:51 Ryan – “I just keep thinking about the Redshift product team. Like, they must be just devastated because clearly these were made for mainstage announcements. It’s even got generative AI. They did all the things and they still didn’t make it.” 23:31 Amazon CloudWatch now monitors EBS volumes exceeding provisioned Performance New Amazon CloudWatch metrics for monitoring I/O latency of Amazon EBS Volumes Amazon ElastiCache for Valkey adds new CloudWatch metrics to monitor server-side response time Cloudwatch will now monitor EBS volumes exceeding provisioned performance! About time! This will allow you to quickly identify and respond to latency issues stemming from under provisioned EBS volumes that may impact the performance of your applications. You can now get two new Cloudwatch metrics for your EBS volumes, including VolumeAvgReadLatency and VolumeAvgWriteLatency, to monitor the performance of your EBS volumes. And finally, Elasticache for Valeky node based clusters now support server side write request latency and read request latency metrics. None of these would have made the main stage, but they’re definitely quality of life improvements. So, thanks AWS. 26:40 Unlock the potential of your supply chain data and gain actionable insights with AWS Supply Chain Analytics In a sign that AWS supply Chain is not getting deprecated anytime soon, they are announcing the GA of AWS Supply Chain Analytics powered by Amazon Quicksight . The new feature helps you to build custom report dashboards using your data in AWS Supply Chain. With this feature your business analysts or supply chain managers can perform custom analysis, visual data and gain actionable insights for your supply chain management operations. Justin, being the executive among us, really appreciated the pretty graphs. 27:38 Amazon Aurora PostgreSQL Limitless Database is now generally available Amazon is announcing the GA of Amazon Aurora PostgreSQL Limitless Databases , a new serverless horizontal (sharding) capability for Aurora. You can scale beyond the existing Aurora Limits for write throughput and storage by distributing the database workload over multiple aurora writer instances while maintaining the ability to use it as a single database. This was previewed last year at Re:invent 2023. 28:44 Justin – “ So one of the things that will mess people up a little bit is that they, you know, way you size this as minimum and maximum capacity measured by Aurora capacity units, which, know, is magic numbers that they created that sort of represent CPUs and things. And so you can set up your minute, your, 16 ACUs as your minimum, and then you can go up to as many as 6,144 ACUs as the maximum, which, that seems like a lot of shards.” 29:48 Amazon SES adds inline template support to send email APIs AWS continues to fix SES annoyances and eliminate platform toil. SES now allows customers to provide templates directly within the sendbulkemail or sendemail API request. SES will use the provided inline template content to render and assemble the email content for delivery, reducing the need to manage template resources in the SES account. We remember Justin asking for this 47 years ago, but it’s here, finally. So, yay? 31:49 AWS announces UDP support for AWS PrivateLink and dual-stack Network Load Balancers AWS is launching UDP protocol support on AWS Privatelink on IPv4 and IPv6 and on the Network Load Balancer over Ipv6. Previously, AWS Privatelink only supported TCP, while NLB supported UDP only over IPv4. This enables customers who use AWS Privatelink and clients that use IPv6 to access UDP-based applications such as media-streaming, gaming, VOIP and other applications. 33:09 AWS AppSync launches new serverless WebSocket APIs to power real-time web and mobile experiences at any scale AWS is launching AWS AppSync Events, a new solution for building secure and performant serverless WebSocket APIs to power real-time web and mobile experiences at any scale. AWS AppSync Events let you easily broadcast real-time event data to a few or millions of subscribers using secure and performant serverless WebSocket APIs, without needing to manage connections or resource scaling. 33:19 Justin – “If I knew what AppSync did and I knew what my use case would be for this, I’d probably be really excited about it, but I don’t really know either, so that’s all I’m gonna say about it.” 35:13 Amazon Route 53 announces HTTPS, SSHFP, SVCB, and TLSA DNS resource record support Route53 now supports HTTPS and Service Binding (SVCB) record types, which provide clients with improved performance and privacy. Instead of only providing the IP addresses of endpoints in response to a DNS query, HTTPS and SVCB records respond with additional information needed to set up connections such as whether your endpoint supports HTTP/3, thereby letting supporting clients connect faster and more securely. In addition you can create TLS Authentication (TLSA) records with route 53. TLSA records may be used to associate TLS server certificates or public keys with your domain name, leveraging DNS Security Extensions (DNSSEC). This provides you with a prerequisite component of DNS-based authentication of named Entities (DANE), a protocol frequently used in conjunction with the SMTP to assure secure and confidential mail transport. 36:12 Ryan – “Well, if all problems are DNS, you should just add more complexity, right?” 40:00 How Executives Can Avoid Being Disrupted by Emerging Technologies Ironic from the cloud company being disrupted by AI. Amazon says innovation happens 50x faster than five years ago… and to be good at staying ahead you need to: Anticipate technology trends and Be a bit of a technology fortune teller They give you 5 ways to do this: Engage in Technology monitoring and scouting Create a culture of curiosity and experimentation Use technology road mapping and scenario planning Form external partnerships We are really looking forward to The Cloud Pod Center of Engagement. Details to follow. It will most likely take place at Disneyland; make those park reservations now. GCP 42:03 Mandatory MFA is coming to Google Cloud. Here’s what you need to know Like what Microsoft recently enacted, GCP plans to implement mandatory MFA for Google Cloud in a phased approach that will roll out to all users worldwide during 2025. To ensure a smooth transition, Google Cloud will provide advanced notifications to enterprises and users to help plan MFA deployments. Phase 1: Starting in November 2024: Encourage MFA Adoption Phase 2: Early 2025: MFA required for password logins Phase 3: End of 2025: MFA for federated users 42:26 Ryan – “I am a little nervous about that phase three just because there’s always differences when you do MFA through Federation as I’ve learned through AWS integrations. And so it’s like, I hope that goes smoothly.” 43:59 Powerful infrastructure innovations for your AI-first future Google is dumping money into AI Hardware at an impressive pace and so we get to geek out with some infrastructure! Woohoo! They are announcing Trillium, their 6th generation TPU, is now available to Google Cloud customers in preview Over 4x improvement in training performance Up to 3x increase in inference throughput A 67% increase in energy efficiency An impress 4.7x increase in peak compute performance per chip Double the high bandwidth memory Double the interchip interconnect New A3 and A3 Mega VMs powered by the NVIDIA H100 Tensor Core GPU 2x the GPU to GPU bandwidth, Up to 2x higher LLM inference performance and ability to scale tens of thousands of GPUs in a dense, performance-optimized cluster for large AI and HPC workloads. Support for the upcoming NVIDIA GB200 NVL72 GPUs, with more details coming soon. Titanium , their system used to offload technologies that underpin their infrastructure, has been enhanced to support AI workloads. Titanium reduces processing overhead on the host through a combination of on-host and off-host offloads, to deliver more compute and memory resources for your workloads. And while AI infrastructure can benefit from all of Titanium’s core capabilities, AI workloads are unique in the accelerator-to-accelerator performance requirements. To meet these needs, they have introduced a new titanium ML network adapter that includes and builds on NVIDIA ConnectX-7 NICs to further support VPCs, traffic encryption and virtualization. Hyperdisk ML is now generally available . Hyperdisk ML is their AI-focused block storage service that we announced in April 2024. Now generally available, it complements the computing and networking innovations discussed in this blog with purpose-built storage for AI and HPC workloads. Hyperdisk ML accelerates data load times effectively You can attach 2500 instances to the same volume, and get 1.2tb/s of aggregate throughput per volume, which is more than 100x higher than offerings from major block storage competitors. Shorter data load times translate to less accelerator idle time and greater cost efficiency GKE now automatically creates multi-zone volumes for your data 46:01 Justin – “ …we want you to know it’s coming because other our competitors are going to be offering these, but we also are going to offer them. So we want you to know that, but we don’t know what they’re going to cost or anything about them because Nvidia hasn’t given us any details, but we want to announce first.” 48:45 C4A VMs now GA: Our first custom Arm-based Axion CPU At Next 24, Google announced the Axion processors , their first custom ARM based CPUs designed for the data center. Now they are Generally Available, the first Axion based VM Series, the C4A, with up to 10% better price-performance than the latest generation Arm-based instances available from leading cloud providers. C4a Vms are a great option for a variety of general-purpose workloads like web and app servers, containerized microservices, open source databases, in-memory caches, data analytics engines, media processing and AI inference applications. “Spanner is one of the most critical and complex services at Google, powering products including YouTube, Gmail, and Google Ads. In our initial tests on Axion processors, we’ve observed up to 60% better query performance per vCPU over prior generation servers. As we scale out our footprint, we expect this to translate to a more stable and responsive experience for our users, even under the most demanding conditions.” – Andi Gutmans, VP/GM Databases, Google C4A broadens their general-purpose VM portfolio, and is offered in a range of configurations: Standard 1:4 vcpu to memory High Memory: 1:8 vcpu to memory High CPU 1:2 vcpu to memory “Honeycomb.io helps engineering teams debug their production systems quickly and efficiently. Sampling is a key mechanism for controlling observability costs. For our customers who are running applications on Google Cloud, we have validated that the new Axion CPUs and C4A VMs offer the best price-performance on Google Cloud for running our Refinery sampling proxy to forward only the most important, representative samples to Honeycomb.” – Liz Fong-Jones, Field CTO, Honeycomb 50:10 Justin – “Yeah, that was a weird quote. For our customers that run on a different cloud than us, this works great. OK.” 51:13 Introducing an industry first: application awareness on Cloud Interconnect Google introduced Cross-Cloud Network to transform and simplify hybrid and multi-cloud connectivity, and enable organizations to easily build distributed apps. As organizations modernize their infrastructure, leveraging AI/ML and other managed services, they have adopted Cross-Cloud Network to reduce operational complexity and lower the TCO. The point of Cloud Interconnect was to provide robust, high bandwidth, SLA backed connectivity to google cloud. With Cross-Cloud interconnect they enable dedicated and private connectivity from Google to another cloud provider. Together, they form the foundation for building hybrid and multi cloud distributed apps. Customers have traditionally lacked the capability to prioritize traffic, forcing them to overprovision bandwidth or risk subpar performance during periods of congestion. TO address this need for traffic prioritization, google is introducing application awareness on Cloud Interconnect in preview. Google Cloud is the first major cloud service provider to offer a managed traffic differentiation solution that empowers you to solve the critical challenge of traffic prioritization over Cloud Interconnect . Application awareness enables flexibility with a choice of two policies: Strict priority across traffic classes and bandwidth shared per traffic class. Application awareness on Cloud Interconnect provides multiple business benefits, including: Prioritization of business critical traffic Lower total cost of ownership (TCO) Fully managed, SLA backed solution 52:29 Matthew – “I just wonder how much, how many people actually need this. Like for QOS, like I feel like I’ve really set it up on VoIP and like backups, offsite backups back in the day. Like that was about it…it just feels like the wrong way to manage it.” 55:01 Speed, scale and reliability: 25 years of Google data-center networking evolution We have talked often on this show how important it is to know the principles behind how the hyperscale of your choice is defined. In the case of AWS, they have a strong regional/availability zone isolation model. For GCP, we have talked about their common storage layer and what it enables. This blog post gives you key insights into the design thinking of the 25 year design of Google’s Network. As Google says, Rome wasn’t built in a day, and neither was Google’s network. But 25 years in, they share some of the details of how they started out small and now run the 5th generation Jupiter Datacenter network which now scales to 13 Petabits/Sec of bisection bandwidth. For perspective, this network could support a video call @1.5MB/s for all 8 billion people on Earth. The network evolution has been guided by a few key principles: Anything, anywhere: Our data center networks support efficiency and simplicity by allowing large-scale jobs to be placed anywhere among 100k+ servers within the same network fabric, with high-speed access to needed storage and support services. This scale improves application performance for internal and external workloads and eliminates internal fragmentation. Predictable, low latency: We prioritize consistent performance and minimizing tail latency by provisioning bandwidth headroom, maintaining 99.999% network availability, and proactively managing congestion through end-host and fabric cooperation. Software-defined and systems-centric: Leveraging software-defined networking (SDN) for flexibility and agility, we qualify and globally release dozens of new features every two weeks across our global network. Incremental evolution and dynamic topology: Incremental evolution helps us to refresh the network granularly (rather than bringing it down wholesale), while dynamic topology helps us to continuously adapt to changing workload demands. The combination of optical circuit switching and SDN supports in-place physical upgrades and an ever-evolving, heterogeneous network that supports multiple hardware generations in a single fabric. Traffic engineering and application-centric QoS: Optimizing traffic flows and ensuring Quality of Service helps us tailor the network to each application’s needs. These principles lead to 2015 and Jupiter, the first petabit network with 1.3 Pb/S of aggregate bandwidth by leveraging merchan switch silicon, Clos Topologies and SDN. 2022 they enabled 6 Pb/S with deep integration of optical circuit switching (OCS), wave division Multiplexing, and highly scalable Orion SDN controller. 2023 13 Petabit per second network by enhanced jumper support to native 400G/s link speeds in the network core. The fundamental building block of Jupiter networks now consists of 512 ports of 400GB/s of connectivity both to end hosts and to the rest of the data center, for an aggregate of 204.8 TB/s of bidirectional non-blocking bandwidth per block. 2024 and Beyond. They are charting for the future with the next gen of network infrastructure, for example they are busy working on networking infrastructure needs for the A3 Ultra VMs , featuring NVIDIA ConnectX-7 networking, supports non-blocking 3.2 Tbps per server of GPU to GPU traffic over RDMA over converged ethernet. They will deliver significant advances in network scale and bandwidth, both per port and network wide. Azure 1:00:03 No new Azure DevOps OAuth apps beginning February 2025 Starting Feb 3 2025, Microsoft will no longer accept new registrations of Azure Devops Oauth Apps. This is their first step in sunsetting the Azure Devops Oauth Platform. Going forward they are advocating for you to build apps on top of the Azure Devops REST API to explore the Microsoft Identity platform and registering a new Entra application instead. All existing oauth apps will work until the official end of life in 2026. 1:00:16 Justin – “So run and provision those as quickly as possible so you have them if you’re working in middle of a project before they go away and you have to redo all your work. 1:01:29 Microsoft names Jay Parikh as a member of the senior leadership team Satya Nadella is welcoming Jay Parikh to Microsoft as a member of the Senior leadership team (SLT), reporting to Satya. Jay was the global head of engineering at Facebook (now Meta) and most recently was the CEO of Lacework. His focus will extend beyond technology, which his passion for and dedication to developing people will foster a strong culture and build world-class talent. Jay will be immersed in learning about Microsoft’s priorities and culture, spending time with senior leaders and meeting with customers, partners and employees around the world. They will share more on his role and focus in a few months….. Have to wonder what the long term viability of Charlie Bell is. 24:29 Justin – “…all I can think of is Azure has been beaten up pretty bad on security. Charlie Bell’s been there about two years, hasn’t seemed to move the needle and I don’t know, but if I was a betting man, I’d say the former CEO of a security startup is probably going to maybe be in charge of something security wise.” Closing And that is the week in the cloud! Visit our website, the home of the Cloud Pod where you can join our newsletter, slack team, send feedback or ask questions at theCloud Pod.net or tweet at us with hashtag #theCloudPod