258: To Q or Not to Q – That is the Question (But, Will We Get a Good Answer?)

Welcome to episode 258 of the Cloud Pod podcast – where the forecast is always cloudy! This week your hosts Justin, Matthew, and Jonathan dig into all the latest earnings reports, talk about the 57 announcements made by AWS about Q, and discuss the IBM purchase of HashiCorp – plus even more news. Make sure to stay for the aftershow, where the guys break down an article warning about the loss of training data for LLM’s. Titles we almost went with this week: Terraform hugs to Big Blue (Bear) The CloudPod hosts again forgets to lower their headphone volume AWS fixes an issue that has made Matt swear many times Google gets mad at open-source Azure has crickets HashiCorp’s Nomadic Journey to the IBM Oasis ‍It’s Gonna be Maaay! A big thanks to this week’s sponsor: Check out Sonrai Securities’ new Cloud Permission Firewall. Just for our listeners, enjoy a 14 day trial at https://sonrai.co/cloudpod General News 01:48 It’s Earnings TIme! Alphabet (Google) Alphabet beat on earnings and revenue in the first quarter, with revenue increasing 15% from a year earlier, one of the fastest growth rates since 2022. They also announced its first dividend and a $70 billion dollar stock buyback. Using layoff money for something other than a buyback? IN THIS ECONOMY? Revenue was 80.54 Billion vs 78.59 expected, resulting in earnings per share of 1.89. Google Cloud Revenue was 9.57B vs 9.35 B expected. Net income jumped 57% to 23.66 B up from 15.05B a year ago. Operating income of the cloud business quadruped to 900M, showing that the company is finally generating substantial profits after pouring money into the business for years to keep up with AWS and Azure. 03:54 Justin – “Yeah, I mean, they’re doing pretty well… I think AI is helping them out tremendously in this regard. I believe it includes G Suite as well. But I mean, like I don’t know how much revenue that is comparatively, but your Google cloud is definitely the majority of it, I think at this point..” 04:20 Microsoft MSFT fiscal third quarter results exceeded on the top and bottom line, but revenue guidance came in weaker than expected. Consensus estimate said Q4 should be 64.5B but Microsoft CFO called for 64B. Revenue grew 17% year over year in the quarter, net coming was 21.94B up from 18.30 billion. Micosoft said that currently near term AI demand is higher than their available capacity, and is focusing on buying more Nvidia GPU units. Azure Revenue and other cloud services grew 31% up from 30% in the previous quarter. Overall Intelligence cloud revenue was 25.71 B up 21% from the year before. Github Copilot apparently has 1.8 Million paid subscribers now – all writing terrible code. So that’s awesome. 05:37 Jonathan – “I wonder how many of those (Copilot subscribers) will go away though. I think a lot of people sign up for things just to check them out and maybe won’t renew them in the long term. I’d be curious.” 06:25 Amazon Amazon Revenue came in at 143.3 BIllion vs 142.5 expected. AWS came in at 25B vs 24.5 Billion expected. Amazon expects a continued jump in profitability for the second quarter but at a measured pace. AWS Sales accelerated by 17% in the first quarter, higher than the 12% by analysts. Over the last year growth in AWS has slowed as business trimmed their costs. This makes AWS a 100B dollar run rate business. Operating income soured more than 200% in the period to 15.3 billion far outpacing revenue growth, a sign that its cost-cutting measures and focus on efficiency is improving its bottom line. AWS accounted for 62% of that profit. Margin on AWS Cloud division was the highest ever at 37.6% The only bad news is that the advertising unit saw a 24% surge to 11.8B meaning its growing faster than AWS. This hurts the store experience IMO. 07:43 Justin – “I mean, you’re basically paying for search placement. So when you search for, you know, binkies for your baby, you know, someone paid for an ad on that for their more expensive item. And then, you know, basically they’re giving you a bunch of listings that are more expensive than what Amazon would have sold to you directly. And you pay more and get a lesser quality product than you would have maybe gotten if you bought directly from Amazon. So I’m not, I’m not a huge fan of that model, but it’s making them a lot of money.” 09:38 HashiCorp joins IBM to accelerate multi-cloud automation Hashicorp announced on Friday that they have signed an agreement to be acquired by IBM “to accelerate the multi-cloud automation journey” they started 12 years ago. Armon wrote this particular blog post, so it has a lot of fond memories of starting the company with Mitchell Hashimoto, etc. Armon’s post says they will continue to build products and services as Hashicorp, and will operate as a division within IBM software. By joining IBM, Hashicorp products can be made available to a much larger audience, enabling them to serve more customers and users. IBM is buying the company for 6.4B, which is a pretty small sum compared to their IPO. Fintan Ryan and Forrest Brazeal had some good insights on the topic below. 10:33 Jonathan – “So I have a take on it, which I haven’t seen anybody else mention yet. And given that IBM already bought Red Hat five years ago or something, and they have the OpenShift and OpenStack ecosystems, I actually think that Nomad, the least understood product in the suite probably, may be kind of a motivator for IBM to buy this. Because I think Nomad addresses some gaps in the container ecosystem of OpenShift, especially when you start to think about IBM’s sort of focus on hybrid cloud.” 17:22 On IBM acquiring HashiCorp Fintan is the director of Market Insights @ github, but previously was an analyst at Gartner and Redmonk Fintan points out that Hashi had a high dependence on a subset of customers with over 100K in ARR. This represented 19% (830 of 4392 customers as of Q1FY24) of their customer base, with majority of the sales being from the US (71%) Over the last three quarters, the rate of growth has slowed, and revenue concentration has remained the same. In addition, client Net Dollar Retention continued to decline, with a very substantial drop in the last two quarters to 115. This is a pretty fast drop, even against the macroeconomic environment. Simply, it was rapidly slowing and couldn’t support its current valuation nor its IPO valuation. Interestingly the BSL change seems to have further hurt them, with the growth dropping to 1.% quarter on quarter immediately after the BSL change, coupled with the negative headlines driven by the Cease and Desist with OpenTofu. It will be interesting to see how IBM sees things with Terraform and OpenTofu, and with their strong support of OSS will there be a change. Fintan says it will matter about where Hashi ends up inside of IBM Two options from his analysis: Bring Hashicorp into Redhat *or* Run Hashi as part of the IBM Cloud Division. So Far IBM is signaling that this will be an independent division, but I suspect that will change over time. Terraform will not help IBM Cloud grow, so it doesn’t make a lot of sense there. It will be interesting if it moves under Redhat and gets combined – or tightly integrated with – Ansible which could be an excellent middle ground before just dying inside of IBM. 19:41 Good Tech Things: Why didn’t one of the big clouds buy Hashicorp? Forrest Brazeal asked and attempted to answer the question I first asked, why didn’t Google or any other cloud provider buy Hashicorp. Forrest points out the reasons why it seems a match made in heaven with Google and Justin had some of these same thoughts. Google is #3 provider, a huge slice of their competitors use and love Hashicorp products everyday. Seems like a way to get new fans. Google has historically cultivated a generous open attitude toward open source. They wouldn’t have thought twice about donating Terraform to the Linux Foundation and reuniting the renegades from opentofu. Google cloud has a long history of making strategic purchases like stackdriver, mandiant and chronicle Google cloud already treats terraform pretty much as their default deployment option. (Just ignore Google Cloud Deployment Managers… no one uses that.) They could easily spend 6.4 B dollars. Forrest goes on to explain why it doesn’t make sense to buy it. Why buy the cow when you can get the milk for free? Google is already getting 100% of the value of Terraform with the OSS version today. They even built their own terraform service called Infrastructure Manager (which Hashi wasn’t too happy about by the way). 6.4 B is just the beginning, you would need to migrate them from AWS. You need to sell Hashi to enterprise companies and build a sales and support business around that, as well as you’re already spending money on Google Cloud Terraform Providers… and oh some of your biggest competitors have co-maintained terraform providers (AWS.) Do you really want to be in the Hybrid Cloud Deployment business? Google doesn’t, they support hybrid as an on-ramp to their services. Forrest rightfully points to the last public filings where Hashi hasn’t figured out how to be a SaaS company and they weren’t confident in their ability to become a services business. Brand Value and OSS Goodwill aren’t enough, and as they recently fired all their python maintainers (will cover in google section) its hard to see them being worried about goodwill. 21:54 Matthew – “I mean, the problem is since the tool is designed to support all the different vendors, it’s hard to have any one vendor buy them. And that’s kind of the problem is they were in this ground of they were trying to help everyone and therefore it’s hard for all of them to get help from all the cloud vendors.” AWS 27:31 AWS supports dynamically removing and adding auto assigned public IPv4 address Amazon VPC announced a network interface setting to dynamically remove and add an auto-assigned public IPv4 address on Ec2 instances. With this capability, customers can no longer require an auto assigned public IPv4 address on their EC2 instance can remove the public IPv4 address, and if needed attach back a new public IPv4 address, by modifying the public iP setting on the network interface. Before today, once a public IPv4 address was auto assigned to an EC2 instance it was not possible to remove it. Want to check out the user guide? You can find it here . 31:16 Amazon Q Business, now generally available, helps boost workforce productivity with generative AI Amazon Q Developer, now generally available, includes new capabilities to reimagine developer experience AWS Announces General Availability of Amazon Q, the Most Capable Generative AI-Powered Assistant for Accelerating Software Development and Leveraging Companies’ Internal Data Dear Amazon…you look desperate. You don’t need to divide the General Availability of Q into 2 Full AWS Blog posts for business and developer, and then also publish a formal press release. We get that you’re doing AI… don’t overplay your hand! We also recognize that you’re presenting earnings this afternoon – and were hoping for positive momentum. But GEEZ. And on that note… AWS is announcing the general availability of the worst feature they’ve ever inflicted on the AWS Console: Q. Specifically, the Business and Developer versions of Q. Amazon Q is designed to make it easier for employees to get answers to questions across business data such as company policies, product information, business results, code base, employees and many other topics by connecting to enterprise data repositories to summarize the data logically, analyze trends, and engage in dialog about the data. AWS is also introducing Q Apps, a new and powerful capability that lets employees build generative AI apps from their companies data. Employees simply describe the type of app they want, in natural language, and Q apps will quickly generate an app that accomplishes the desired task, helping them streamline and automate their daily work with ease and efficiency. Amazon Q Developer is designed to help developers, with 30% of developer time spent on coding and the rest spent on tedious and repetitive tasks… cough *meetings* cough. Q is here to help developers and IT professionals with all their tasks — from coding, testing and upgrading applications to troubleshooting, performing security scanning and fixes and optimizing AWS resources. Amazon claims it has the most accurate coding recommendations by making suggestions in near real time. Amazon Q developer has the highest reported code acceptance rates in the industry for assistants that perform multi-lien code suggestions, with BTG group recently sharing that they accepted 37% of Q code suggestions and National Australia Bank reported 50% acceptance rates. Q also has the ability to customize by leveraging the customers internal code base to provide more relevant and useful code recommendations. Amazon Q developer agents will perform a range of tasks from implementing features, documenting and refactoring code to performing software upgrades. Developers can simply ask Q to implement an application feature such as asking to create an “add favorites” feature in a social sharing app, the agent will analyze the code and generate a step-by-step implementation plan. Best in class Security vulnerability and remediation Q is an expert on AWS and optimizing your AWS environment. The interface is available where you need it AWS console, In Slack, or in IDE’s such as VS code and Jetbrains. Amazon Q for Business allows your employees to get access to the wealth of information shared and stored in your internal repositories. Q Unites more data sources than any other generative AI assistant available today with 40+ commonly used business tools such as Wikis, intranets, atlassian software, Gmail, exchange, salesforce, servicenow, slack and S3 . Built from the ground up with security and privacy in mind Inventive generative BI allows analysts to build detailed dashboards in minutes and business users to get insights fast First-of-its-kind capability that helps every employee go from conversation to generative AI-powered App in seconds. 32:55 Justin – “So that’s scary sounding to me, that employees are just creating apps with our data, and you’re just hoping it’s not gonna lie or do things… So it’s doing great, doing really good, super happy about Q. And I definitely would not trust it with my employee, my internal company data, I don’t think at this point.” 33:34 Matthew – “I was just gonna say, I feel like that’s the issue with all of them. It’s like, how much do you trust any of these providers with all your data and making sure that only the right people get access to the right subset of that data? So your finance guy doesn’t accidentally gain access to all of HR by asking the right questions. That’s kind of always meant to worry with a lot of these things – or just start making stuff up.” Listener Note: Anyone out there have any real-world experience with Q? We’d love to hear it. Hit us up on our Slack channel, or send Justin an email. Justin@thecloudpod.net GCP 41:05 Introducing new ML model monitoring capabilities in BigQuery Monitoring ML models in production is now as simple as using a function in BigQuery! Today Google is introducing a new set of functions that enable model monitoring directly within BigQuery. Now, you can describe data throughout the model workflow by profiling training or inference data, monitor skew between training and serving data, and monitor drift in serving data over time using SQL for BigQuery ML models as well as any model whose feature training and serving data is available through BigQuery. With these new functions, you can ensure your production models continue to deliver value while simplifying their monitoring. 41:40 Jonathan- “Those are some really useful features. And I think it’s going to just go over most people’s heads because they have no concept of what the benefits of these things actually are. So that really the whole point of monitoring the skew between training and the sort of production data sets is that as your customers start to do different things with your models, if the things they’re doing are no longer represented accurately by the training set, then you need to retrain.” 44:52 2024 DORA survey now live: share your thoughts on AI, DevEx, and platform engineering Hola explorers! The 2024 Dora survey is live! Is your team coming up with rapid change? Are you able to meet customer expectations while delivering value and maintaining a healthy team? Take 15 minutes to complete the 2024 Dora Survey . The three key areas of learning for this years Dora report: Artificial Intelligence (AI) Platform Engineering Developer Experience The DORA results will come out later this year and we’ll talk about them when they are released. Interested in last year’s results? You can find them here . 47:02 Python, Flutter teams latest on the Google chopping block Google’s latest round of layoffs hit engineering working on Flutter and Python. The python team was reduced in favor of a new team based in Munich. One of the Hacker news articles talked about one of the laid off engineers (zem) who wrote what they were responsible for as part of the python team: https://news.ycombinator.com/item?id=40176338 47:49 Justin – “Google and many other companies are rapidly reassessing where their talent is, how much their talent costs, and where in the globe that talent is located. And so why sad? I don’t think it’s the end of the world, but definitely Google is not the same company it was five years ago.” 48:36 In-context observability with customizable dashboards everywhere on Google Cloud You can now tailor and customize the dashboard to your unique needs in the c context of the services, including popular requests like adding/removing charts, adding raw logs and changing the configuration of the charts in Cloud Monitoring With this capability, you no longer need to hop between different observability solutions to get the signals you need for troubleshooting and remediation. Customizable dashboards are available for GKE, Compute Engine, Cloud Run, Cloud Functions, Cloud Storage, Dataproc, Dataflow, MySQL System Insights and other Google Cloud Services. 21:25 Justin – “I’m super glad about this. Um, you know, cause this is my frustration with CloudWatch. You know, you go into RDS and you are looking at a database. You’re like, Oh, I want to see that chart differently. And you can’t really customize it inside of RDS. You have to go into cloud watch and then you make all your modifications and cloud watch. Um, but they’re not linked together. And so I liked it, this is a nice enhancement to be able to do that customization, right. And the service you need is tied to your user. And I think you also can publish these dashboards to others as well. So, you get kind of the best of both worlds.” Aftershow 50:41 Are We Running Out of Training Data? The information (sorry paywall) had an article by Brad Kenstler where he commented on the AI Index report about the fact we are running out of high-quality language data needed for training AI and will be out by end of year. That could be an issue for LLMs that have grown by coming in more and more data with larger computing power. THis may result in LLM hitting a quality wall. It’s hard to say how bad this is as we don’t know the training source and data set of most of the LLM’s today. The first part to consider is not all data needed is high quality, for instance a customer support system where a lot of the data is tickets with “I ordered the wrong size” vs complex issues where you have to really think about an answer. LLMs can solve the simple problems with only a few sources of data, whereas the more complicated solution is more beneficial to the model because it’s rare. This will likely lead to a data monetization gold mine of private data that isn’t available publicly to help build systems which is where Retrieval Augmented Generation is a huge boon. Bigger issue may be all the low content drivel produced by these AI models polluting the internet. 54:08 Jonathan – “I think the prevalence of English content over content in other languages will definitely put the speakers of those languages at a disadvantage. I know that the quality of the model is very dependent on the amount of data it’s trained on.” Closing And that is the week in the cloud! Go check out our sponsor, Sonrai and get your 14 day free trial. Also visit our website, the home of the Cloud Pod where you can join our newsletter, slack team, send feedback or ask questions at theCloud Pod.net or tweet at us with hashtag #theCloud Pod