226: Duet, Co-Pilot, and a Code Whisperer Walk into a bar in San Francisco

Welcome episode 226 of the Cloud Pod podcast – where the forecast is always cloudy! This week Justin, Matt and Ryan chat about all the news and announcements from Google Next, including – surprise surprise – the hot topic of AI, GKE Enterprise, Duet, Co-Pilot, Code Whisperer and more! There’s even some non-Next news thrown into the episode. So whether you’re interested in BART or Bard, we’ve got the news from SF just for you. Titles we almost went with this week: ️The cloud pod sings a duet, guess who was singing You get AI, you get AI, Everyone Gets AI Does a Mandiant Hunt, Or does a Hunter mandiant? ️The Cloud Pod goes into ROM Mode Does a mandalorian Hunt, Or does a Hunter a mandalorian? A big thanks to this week’s sponsor: Foghorn Consulting provides top-notch cloud and DevOps engineers to the world’s most innovative companies. Initiatives stalled because you have trouble hiring? Foghorn can be burning down your DevOps and Cloud backlogs as soon as next week. General News this Week: 01:23 Introducing Code Llama, a state-of-the-art large language model for coding So you know Github Copilot, Duet AI, and Codewhisperer…. But do you know Code LLama? (Meta you better get good stickers on this) Meta has released the source code for the Llama 2 based Code Specialized LLM in three sizes 7B, 13B, and 35B parameters. Each model is trained with 500b tokens of code and code-related data. The 7B and 13b base and instructor models have also been trained with fill-in-the-middle capability allowing them to insert code into existing code. The 7B model can run on a single GPU, the 34B model however returns the best results and for the best for coding assistance… while the 7b and 13b are great for real-time code completions. Training recipes for Code Llama are available on the Github Repository . 04:08 Matthew – “It’s interesting; if you go deep into the article there, they start to digress into like ‘Hey, this 7 and the 13 billion are better for near real time response back’ and the 34 billion… is better for fine tuning for yourself. So they really go into a little bit more detail of how to do it. And, you know, I think they also put out some code snippets if you kind of dive into it a little bit more, which I thought was very nice.” 05:32 OpenTF Announces Fork of Terraform Remember when we talked about Open TF’s manifest begging HashiCorp to backtrack on adopting a BSL license? Well guess what? HashiCorp didn’t listen. Insert sad sound effect. In response, OpenTF has officially forked Terraform. They hope to have the repository available to you within the next 1-2 weeks, with their goal to have an OpenTF 1.6 release. Want to keep up with their progress? They’ve created a public repository where you can track their progress. Check that out here . 06:37 Vlad Ionescu Open TF is a Joke Some opinions are not as keen on this and think it’s a huge distraction and waste of time. We will definitely be following up on this in the next three months, so stay tuned. 07:39 Ryan – “It’s gonna be hard to get community support to drive a fork for something as large as Terraform. I agree that it’s going to be a challenge. I don’t know if I agree with anything else. I was trying to read the thread and trying to sort of not have a visceral reaction to the tone and it’s just, it’s while complaining about drama in the most dramatic way possible. It sort of defeated the purpose for me and I was like I was looking for a little bit more insight because I haven’t. I go very deep and I haven’t really formed an opinion on if this is a good idea or not. Yeah, I don’t know. I think the jury’s still out and continue to watch and see if it takes.” 11:44 Matthew – “I’m sure you guys remember when elastic search moved over, they like made it so that you can’t connect with a non elastic search connector to it. And like, I’m just kind of envisioning that happening where like AWS, you know, Terraform AWS updates their provider to not only allow this, and it becomes a chicken and egg… you did this, we do this. I feel like Palm OS did it with iTunes back in the day… we connected to iTunes and then Apple blocked it. Like I just don’t see where this is gonna go for them.” AWS 13:32 Create Write-Once-Read-Many Archive Storage with Amazon Glacier If you’re at all interested in mistakes that will cost you a lot of money for a very long time, well look no further! AWS is introducing a new glacier feature that allows you to lock your vault with a variety of compliance controls that are designed to support this important record retention use case. Once locked, the policy cannot be overwritten or deleted. Glacier will enforce the policy and will protect your records according to the controls. Creating the wrong policy can make your data undeletable for a long time. Don’t say we didn’t warn you. 15:37 Announcing Amazon Managed Service for Apache Flink Renamed from Amazon Kinesis Data Analytics Did you know Amazon had a managed Flink service? Neither did we! In what I hope will be an awesome change in pace for AWS, they are renaming Amazon Kinesis Data Analytics to Amazon Managed Service for Apache Flink. This relates to a tweet from Ben Kehoe about that people didn’t know AWS had a managed Flink service, and someone tweeted “Wait they have a managed Flink offering?”. (So at least we weren’t the only ones.) The cute names are fun, but also make it difficult to discover things. 17:10 AWS Compute Optimizer now supports licensing cost optimization for Microsoft SQL Server AWS Compute Optimizer now supports licensing cost optimization for SQL Server. Making recommendations like downgrading your SQL server edition to standard or BYOL licensing. These seem like really dumb recommendations without a lot more understanding and context. GCP 19:18 Welcome to Google Cloud Next ’23 AI Was the theme of the day. Everywhere… I couldn’t escape hearing about AI Literally. AI. All day. Everyday. Impressions of the TK/Sundar Keynote (Forced smiles for the win!)? How about going to the DMV? Event/Sessions/Venues. Announcements 23:15 New Titanium backed hardware will allow faster processing of machine learning and AI capabilities. 23:48 Cloud TPU V5e – Most cost efficient, versatile and scalable purpose built AI accelerator to date. Now customers can use a single cloud TPU platform to run both large scale AI training and inference. (Point to Jonathan!) 24:12 A3 Vms with NVIDIA H100 GPU to receive better training performance over prior generation 2 24:53 GKE Enterprise (formerly known as Anthos in many ways) Enables Multi-cluster horizontal scaling plus GKE features like autoscaling, workload orchestration, automatic upgrades and now available with the Cloud TPU V5e GKE Enterprise edition includes: A new multi-cluster feature (“fleets”) Managed security features A fully integrated and fully managed platform Hybrid and multi-cloud support GKE Enterprise edition includes: Group similar workloads into dedicated clusters Apply custom configurations and policy guardrails Isolate sensitive workloads Delegate cluster management Spend less time managing the platform Run container workloads anywhere 28:42 Ryan – “And it is still the Anthos we’ve grown to love, right? So it’s still a huge multi-cloud or hybrid cloud opportunity for a ton of people and companies, right? So that they can have sort of a consistent experience to offer across their data centers and any one of the cloud hypervisors. So it’s pretty cool there too.” 29:37 Matthew – “I’d be curious to see and I don’t know if you ever would, but like Google put out, Hey, this main number, you know, this percentage of customers actually using it for multi-cloud versus multi-cluster versus, you know, how are people actually leveraging the enterprise product?” 31:04 Cross-Cloud Network – a global networking platform that helps customers connect and secure applications across clouds. It is open, workload optimized and offers ML-powered security to deliver zero trust. Reduces cross cloud network latency by 35% Three key tenants: Open, Secure and Optimized Allows you to address distributed applications, secure access for hybrid workforces and deliver internet facing apps Cross Cloud Interconnects support Alibaba Cloud, AWS, Azure and OCI. “Yahoo Mail is moving its backend onto Google Cloud and leveraging the planet-scale network for high performance and secure access to Google’s data services. Cross-Cloud Network and Interconnects for high-scaled and high-performing secure access to Spanner and BigQuery will help Yahoo deliver performance and security across hundreds of millions of mailboxes.” – Aaron Lake, Senior Vice President and CIO, Yahoo 32:18 Ryan – “When I worked for Yahoo, they were very heavy, a data center company. And that was my primary role was automating a lot of that inner company sort of play for launching infrastructure. So building our own versions of AWS and Azure services at the time. And so to see them, you know, taking advantage of the public ones is great, right? Because they’re at a scale that is gonna make the product. really good for everyone else.” 33:29 Global Access and Global Backends allow you to private clients from any region to access internal load balancers in any google cloud region. And Global Backends allow internal ALB to health-check and send traffic to globally distributed backend services To simplify the network layer, VPC Spokes support in Network Connectivity Center now lets you smoothly scale VPC connectivity, providing reachability between a large number of VPC spokes. Peered VPC spokes with overlapping RFC1918 addressing will be able to utilize Cloud NAT’s Inter-VPC NAT feature , ensuring that Inter-VPC network traffic stays within the Google Cloud network versus traversing the internet to help ensure privacy and security. Cloud applications now support cross-project service referencing plus support for MTLS Cloud NGFW in preview, a cloud first next gen firewall powered by PAN. Provides inline threat protection with 20x higher efficacy compared to other cloud firewalls, a built in distributed firewall architecture, unified network security posture controls and simplified single-policy threat response. 37:24 Google Distributed Cloud is being expanded to support new Vertex AI integrations and a new managed offering of AlloyDB Omni on GCD-hosted. If you’ve ever had the need to run Postgres AlloyDB, you can now do that in your data center on Google Managed Hardware. 37:58 Vertex AI got lots of love at the conference – tons of new goodies. Palm 2, Image and Codey Upgrades New tools to tune Palm2 an Codey New Models with Llama 2 and code llama, as well as Technology Innovative Institutes Falcon LLM, a popular open source model, as well as pre-announced Claude 2 from Anthropic. Vertex AI extensions will allow developers to access, build and manage extensions that deliver real time information, incorporate company data, and take action on the users behalf. This allows Vertex to take action on third party systems like CRM. Enterprise Grounding service Digital Watermarking on Vertex offers a technology powered by Google DeepMind SynthID, offering a state of the art technology that embeds the watermark directly into the image of pixels, making it invisible to the human eye and difficult to tamper with. Colab Enterprise to allow ease of use of Google’s Colab notebooks with Enterprise level security and compliance. 39:44 Duet AI Duet AI in Google Meet and Google Chat are now available. Duet AI for BigQuery provides contextual assistance for writing SQL queries. Duet AI for GKE and Cloud run provides gen AI assistance to cut down on the time it takes to run containerized apps. Duet AI in Spanner, Alloy and Cloud sql, helps you generate code to structure, modify, or query data using natural language. As well as their bringing Deut AI Database Migration Service to help you automate the conversion of database code such as stored procedures, functions, triggers and packages Duet AI also comes to Security in Chronic Security Operations, Mandiant Threat Intelligence and Security Command Center. 32:18 Ryan – “I know what I want, but I do not know the SQL syntax to get what I want. And this is, it is a fantastic feature that I’ve played around with a little bit for a couple hours. And I, like, I will never ever write SQL any other way.” 42:49 Justin – “And I’m actually right now trying to backup an RDS MySQL database. And if I could use this to figure out how to make that better. Cause I foolishly thought I’d save time by using MySQL command center or the management center cause we have it set up for this particular database, and I regret everything about it.” 45:38 Analytics Big Query Studio is a single interface for data engineering, analytics and predictive analysis, which will increase efficiency for data teams. Plus it integrates into vertex AI foundation models (still gotta get that AI in there.) Alloy DB AI offers an integrated set of capabilities for easily building GenAI apps, including high performance, vector queries that are up to 10x faster than standard postgres. 46:33 Security Mandiunt Hunt for Chronicle to integrate the latest insights into attacker behavior from Mandiant’s frontline experts with Chronicle Security Operations. Agentless vulnerability scanning: this posture management capability in security command center detects operating system, software and network vulnerabilities on compute engine virtual machines Cloud Firewall Plus adds advanced threat protection and next-generation firewall capabilities to their distributed firewall service powered by PAN. Assured Workloads now support the Japan region for Japanese requirements on encryption keys and administrative access transparency. Predictions Ryan Lucas Generative AI Prediction – Finops Practice and Cost Management AI solution A networking feature that only supports IPV6 Jonathan Baker TPU V5 (Super Computer in a Box) Generative AI for Contact Center and/or Retail AI Justin Brodley Google Bard in workspaces will be GA Not going to announce anything New. Matt Kohn Bard via API Additional Security tooling Ci/CD Announcement that they Gain Market Share by @ least 5% Some sort of competitor for AWS Lattice Tie Breaker: Justin: 6 Jonathan: 9 Ryan: 1 ⭐Gold Star to Jonathan⭐ Azure 57:33 Generally Available: Trusted launch as default for VMs deployed through the Azure portal Trusted launch hardens your Azure VM with security features that allow administrators to deploy virtual machines with verified and signed bootloaders, OS kernels and boot policy. We aren’t really sure why this took so long, but better late than never. 1:01:01 Generally available: Azure Container Apps jobs Jobs – a new container app feature previewed at Build is now GA. Azure Container App jobs support three trigger types: Manual, Scheduled and Event Driven. Manual Jobs are triggered by a user or an external system, such as another container app. Common scenarios for jobs include: Running a one time containerized data migration job, running a scheduled recurring containerized batch job, such as a nightly inventory processing job, Running a containerized job in response to an event, or running a CI/Cd build process such as Azure Pipelines agents and github action runners. 1:01:49 Justin – “If you’re into container app jobs and using these to do kind of your scheduled tasks, this is nice, I actually like this feature.” Closing And that is the week in the cloud! We would like to thank our sponsors Foghorn Consulting. Check out our website, the home of the cloud pod where you can join our newsletter, slack team, send feedback or ask questions at thecloudpod.net or tweet at us with hashtag #thecloudpod