266: AWS Billing Finally Comes into FOCUS

Welcome to episode 265 of the Cloud Pod Podcast – where the forecast is always cloudy! It’s a full house this week – Matthew, Jonathan, Ryan and Justin are all here to bring you the latest in cloud news – including FOCUS features in AWS Billing, Magic Quadrants, and AWS Metis. Plus, we have an Andoid vs. Apple showdown in the Aftershow, so be sure to stay tuned for that! Titles we almost went with this week: Tech reports show Gartner leads in the BS quadrant ‍⚖️Oracle adds cloud and legal expenses to their FinOps hub AWS Metis: Great chatbot, or Greek tragedy waiting to happen? The cloud pod rocks Cargo Pants A sonnet is written for FOCUSing on spend A big thanks to this week’s sponsor: We’re sponsorless! Want to reach a dedicated audience of cloud engineers? Send us an email, or hit us up on our Slack Channel and let’s chat! General News 01:40 Finops X Recently Justin attended FinOps in beautiful and sunny San Diego – and if you weren’t there, you really should plan on attending next year. This year’s topics included: Focus 1.0 State of Vendors Conference size – they will most likely outgrow this particular conference center, seeing as how they’re either selling out or pretty close to it. Coolest thing about the conference – on stage all the biggies – TOGETHER. It’s great to see them all together talking about how they’re making Finops better, and introducing new things for Finops and not just saving them for their own conferences. Next Year – Is Oracle going to be on stage next year? 08:22 Justin – “The shift left of FinOps was a big topic. You know, how do we get visibility? How do we show people what things are going to cost? How do we make sure that, you know, people are aware of what they’re doing? And so I think, you know, it’s just a recognition that is important and just as important as security is your cost. And in some ways security is part of your cost story. Because if you bankrupt your company, that’s a pretty bad security situation.” 10:17 Introducing Managed OpenSearch: Gain Control of Your Cloud with Powerful Log Analysis Listen. We don’t really *care* about OpenSearch – but the reality is it’s taking over the world. Nobody is doing ElasticSearch anymore. Digital Ocean is launching Managed OpenSearch offering, a comprehensive solution designed for in depth log analysis, simplifying troubleshooting, and optimizing application performance. With Digital ocean you can Pinpoint and analyze log data with ease, customize log retention, enhance security and can scale with your business and receive forwarded logs from multiple sources including Digital Ocean droplets, managed databases, etc. Interested in pricing? You can find that here . Or, if you’d like to take a product tour, you can do that here . 12:11 Ryan – “It’s the important ones where everything revolves around it and so no one touches it. And so they end up getting the worst spaghetti code and it’s old and tacked on. It always is…It took me three months to understand that code reverse engineering, how it works. And now I’ve got it pretty down. So like when it breaks, I could fix it, but I wouldn’t try to refactor it at all.” AWS 13:57 AWS CodeArtifact adds support for Rust packages with Cargo In news that we’re surprised got a whole blog write up… Rust developers can now store and access their libraries (known as crates ) on AWS CodeArtifact . No additional Rust packages beyond the GB stored, requests and data transfer out to the internet or to other AWS regions. 14:16 Ryan – “I’m a fan of never running a software repository again. And so it’s the support of these new features and functions that allows that managed service sort of usage. So I think it’s fantastic.” 18:10 Anthropic’s Claude 3.5 Sonnet model now available in Amazon Bedrock: Even more intelligence than Claude 3 Opus at one-fifth the cost Three months since Amazon announced support for Claude 3 , their back with the first to release Claude 3.5 model family on Bedrock . Claude 3.5 raises the industry bar for intelligence, outperforming other generative AI models on a wide range of evaluations, including Anthropic’s previously most intelligent model, Claude 3 Opus. Claude 3.5 Sonnet key improvements including: Visual processing and understanding Writing and content generation Custom support and natural language processing Analytics and Insights Coding and Software development 18:48 Jonathan – “I played with it, and it wrote some code around first time. First time. Yep. I just, I skimmed through it like, yep, that looks good. Copy paste – works first time. It’s awesome. I also had it generate… so that was, that was the LDAP group management stuff. I had it, I had it right. And then I also had it write a browser based game, Tic Tac Toe in HTML and JavaScript … and it just spat that out like it’s done it a thousand times, which it probably has. But that worked fine. I wanted to tweak it – like its AI that it was playing tic -tac -toe with, so it may be a new version. And then I played it. It was like, no going back twice. Did I win? Yeah, I did.” 20:30 Announcing the general availability of fully managed MLflow on Amazon SageMaker Amazon is announcing a GA of a fully managed MLflow capability on Amazon Sagemaker . MLflow, a widely-used open-source tool, plays a crucial role in helping ML teams manage the entire ML lifecycle. Customers can now effortlessly set up and manage MLflow Tracking Servers with just a few steps, streamlining the process and boosting productivity. MLflow can be used to track multiple attempts at training models as runs with experiments, compare these runs with visualizations, evaluate models, and register the best models to a model registry. There are three core components to the MLflow capability: MLflow tracking server, which is a stand-alone HTTP server serving multiple REST API endpoints for tracking runs and experiments, enabling you to begin monitoring your ML experiments efficiently. MLflow backend metadata store, is a critical part of the MLflow tracking server, where all metadata related to experiments, runs, and artifacts is persisted. MLflow artifact store provides a storage location for all artifacts generated during ML experiments, such as trained models, datasets, logs, and plots. 21:37 Jonathan – “It’s interesting because MLflow for years has been a competitor to SageMaker. In the US, people have just deployed that and used it to manage running experiments and building models and the workflows around that. And so for them to bring it in like this is, it’s both interesting and kind of cool to have it as a managed service now. But no longer a competitor anymore.” 22:46 Rightsizing Recommendations for Amazon RDS MySQL and RDS PostgreSQL in AWS Compute Optimizer At Finops X AWS announced that AWS Compute Optimizer delivers new recommendations for RDS MySQL and PostgreSQL databases. These recommendations help you detect idle RDS instances and identify the optimal instance types and provisioned IOPS settings for your existing RDS DB instances, so you can reduce costs for ideal and over-provisioned RDS DB instances or increase the performance of under-provisioned workloads. To rightsize RDS databases, it takes in multiple variables and can be very time consuming, you need dedicated engineering resources specializing in databases to analyze the cost and performance. To solve this Amazon is giving you the AWS Rightsizing recommendations for RDS. It makes a recommendation in 2 parts, first a focus on DB Instance size and second you will see a recommendation for DB instance storage. 23:30 Ryan – “I was really happy to see that they included the IOPS provisioning in this optimization, just because so many database engineers, they live by that one outage because of drive performance and all the SQL engines say you need to have this and all these things, and then it just sits idle and it’s so much cost. So this is pretty great to see because I think this is one area I know that in most places I’ve worked where there’s a heavy over provisioning.” 26:28 AWS Billing and Cost Management now provides Data Exports for FOCUS 1.0 (Preview) Data Exports from AWS Billing and Cost Management to the FOCUS 1.0 schema to Amazon s3. This feature is in preview. Woohoo. AWS Billing and Cost Management now provides Data Exports for Cost Optimization Hub Data exports for the cost optimization hub now enables customers to export their cost optimization recommendations to S3. 27:38 Report: Amazon developing AI chatbot that would compete with ChatGPT and others According to a report from BI, Amazon is developing a consumer-focused AI chatbot that would compete with OpenAI’s ChatGPT and could be revealed later this year. The new chatbot service, code name “Metis” would be accessed via web browser and powered by a new foundational model. The move would join a growing list of technology companies building their own AI assistants. I assume that this will be built on top of their Titan models. I assume that this is more consumer-facing, and they didn’t feel about the Q . GCP 29:57 GKE under the hood: What’s new with Cluster Autoscaler Google regularly improves the cluster autoscaler, and they wanted to shed some light on these small little wins. Target Replica count tracking, this feature accelerates scaling when you add several Pods simultaneously (think new deployments or large resizes). It also eliminates a previous 30-second delay that affected GPU autoscaling. This capability is headed to open-source so that the entire community can benefit from improved K8 performance. Fast homogeneous scale-up: if you have numerous identical pods, this optimization speeds up the scaling process by efficiently bin-packing pods onto nodes. Less CPU waste: The CA now makes decisions faster, which is especially noticeable when you need multiple scale-ups across different node pools. Memory Optimization: Not directly visible, but the CA has undergone memory optimizations that contribute to its overall efficiency. 31:02 Ryan – “Very cool. I mean, I still see dummy workloads for pod scaling and kind of hackery in order to address this issue with some of my spiky workloads. And so the more they add these improvements, the more we can sort of stop supporting all this stuff. Good. Awesome.” 31:52 Google is a Leader in the 2024 Gartner® Magic Quadrant for Analytics and Business Intelligence Platforms It’s that time again – but that’s mostly because it’s ALWAYS that time. Break out your wizard hats – it’s Magic Quadrant time! Microsoft was the leader followed by Salesforce, but Google, Qlik and Oracle were all on the same level in this year’s quadrant. AWS has climbed a bit too now leading the challengers box and almost at the point to cross to the leaders quadrant. 33:24 Jonathan – “What does Copilot have to do with this thing? 33:28 Justin – “Because you want co -pilot in all your analytics, so you can ask it to make pretty reports of AI and natural language!” 35:42 Google is a Leader in the 2024 Gartner® Magic Quadrant™ for Data Science and Machine Learning Platforms This quadrant looks similar but Databricks, Microsoft, Google and AWS are on top of the magic quadrant. Google: S: Foundational Models Balanced Portfolio Delivery C: Data and AI Governance Platform Ecosystem Core Data Science AWS: S: Security Compliance Custom AI Infrastructure Training and Community C: Core data Science Complexity GenAI Market Sentiment Microsoft: S: Enterprise GenAI Upskilling Research and Development C: Separate Product Lines Data Exploration and visualization Copilot integration 36:42 Matthew – “I just like how SageMaker is like a positive there for AWS. And I’m like, SageMaker has been there for like 10 years, I feel like now. Like it’s been there for a long time. And like before nobody used it and now everyone’s like, ooh, SageMaker. And I’m like, it’s been there for a long time.” 37:19 Leveling up FinOps: 5 cost management innovations from FinOps X 2024 Google also announced some things at Finops X . 1. BigQuery View that shows billing data in FOCUS – FinOps Open Cost & Usage Specification. 2. Speaking in the language of business, not technology with Gemini cloud assist 3. Finops Hub is working through cloud environments for sustainability and carbon. 4. New Scenario modeling for CUDs , with the ability to look at different lookback periods, eliminate data noise (ie black friday), seeing instant results and collaborating with confidence. 5. Send actionable alerts for cost anomaly detection with easy email and pubsub capabilities and slack. 38:08 Justin – “Yeah, that modeling for CUDs was probably the highlight of the announcements from the vendors. You know, just the ability to do things like a lot of different windows. Like Amazon has some look back, so it’s like 30 or 90 days. They have a bunch of different carved out periods. And then yeah, being able to say like, look, this weekend where that outage happened and we spun up a bajillion Kubernetes clusters that kept failing or, you know, Hey, a black Friday event or, you know, those are the things that’s nice to be able to just eliminate that and say, that’s just a data anomaly. Don’t count that into my, my cut analysis as I go through it.” 38:47 Announcing Anthropic’s Claude 3.5 Sonnet on Vertex AI, providing more choice for enterprises Google has also brought Claude 3.5 Sonnet on Vertex AI. And it’s super expensive. That is all. 38:58 Jonathan – “I didn’t check Bedrock, but on Vertex, you pay differently for the input tokens and the output tokens, but effectively since you want it to be generating stuff, because it’s GNI, $15 per million tokens, which is quite pricey. Like the smaller model, the Haiku model, is only $1 .25 per million tokens, so just to put it in perspective.” 39:47 Simplify historical data tracking in BigQuery with Datastream’s append-only CDC Datastream, Google cloud’s serverless CDC service, recently introduced a new feature called append-only mode that streamlines the process of replicating changes from your operational databases to BigQuery. This feature offers an efficient and cost-effective way to maintain historical records and track changes to operational data overtime. In traditional CDC- based replication, when a record in your source database is updated or deleted, the corresponding record in the destination is overwritten, making it difficult to track the history of changes. Append-only mode addresses this challenge by preserving every change as a new row in your target BigQuery table. Each row includes metadata that captures the type of change (insert, update, or delete) a unique identifier, timestamp and other relevant information, which can be used to order and filter the data as needed. Append only mode is particularly beneficial in scenarios where you need to maintain a historical record of changes. Some common use cases include: Auditing and Compliance Trend Analysis Customer 360 Analyzing embedding drift Time travel 41:38 Jonathan – “Well, Google only takes cash, not souls.” Azure 41:53 Finops X announcements Azure is adding a estimate total costs before deploying the console Deeper insights into K8 costs Advisor Workbooks to help optimize costs New and updated FinOps tools including Azure Optimization Engine, Cost Optimization Workbook, and Finops Framework 2024 updates. Export multiple datasets from billing including FOCUS 1.0 compliant data Self Service analytics via Fabric (for those of you paying through the nose for Fabric.) 43:40 Matthew – “Azure really doesn’t like to announce the things that they’re doing, I feel like… Well, or it’s like it’s in private preview, then preview and then public preview and then then something else. Then there’s G. So by the time you get there, you’re like, I’m not interested because I’m not going to use it until it’s G aid or production workloads. So it’s like. That’s it.” Oracle 45:24 More Finops! Oracle also got into the Finops spirit by being on the floor and announcing their new Finops Hub . Businesses can track cloud resource costs more closely than ever. Finops Hub helps practitioners to answer these questions: Are budgets on track Do we have enough credits left so we don’t pay overages What resource groups have the greatest impact on cost Where can I-optimize costs How can I prevent unauthorized spending? 46:55 Jonathan – “Well, budgets are so invisible to most people* in the organization now, I think. And so if they’re building tooling, which makes it put in front of people and shows you what the budget is and shows you where you are on track or not on track, it’s just a useful tool. It’s a nice perspective.” *Except Justin After Show 35:23 After Show: Apple WWDC 2024: the 13 biggest announcements Now is the perfect time for Jonathan to point out that Android has had all of these things for 47 years…but here goes. Apple Intelligence for Iphone, Ipad and Mac (And Elon says screw you apple devices) Siri gets an AI boost Apple is building ChatGPT into Siri New AI Features in Mail, Messages and Photos Iphone gets more customizable in IOS 18 Iphone is getting RCS support (Green vs blue) Apple TV Plus adds “insights” A passwords app to keep track of your logins iPADOS 18 adds a calculator app with pencil support MacOS 15 will let you mirror your iphone to MacOS Apple watch gets surface widgets Vision OS2 adds spatial photos and an Ultrawide Mac Display Airpods Pro will let you quietly respond to Siri Closing And that is the week in the cloud! Visit our website, the home of the Cloud Pod where you can join our newsletter, slack team, send feedback or ask questions at theCloud Pod.net or tweet at us with hashtag #theCloud Pod