Today's post is a fairly short one. Let's say you want to connect to Azure Data Lake Storage Gen2 from SQL Server 2022. We saw in a recent post how to connect to Azure Blob Storage. When it comes to Azure Data Lake Storage, the story is almost the same, though there are a couple…
SQL Server 2022 Data Virtualization: Insert into Azure Blob Storage
Several years ago, I wrote a blog post on how to insert data into Azure Blob Storage from SQL Server using PolyBase. That technique used PolyBase V1: the Java connector for Hadoop. With SQL Server 2022 eliminating that connector, we're going to learn the new method. Starting out the Same Regardless of the version of…
PolyBase and Azure Blob Storage
A long, long time ago (I can still remember) PolyBase used to connect to Azure Blob Storage. Specifically, you could use the wasbs:// protocol and connect to Azure Blob Storage over WebHDFS. Here's an example of an external data source which would work for SQL Server 2016 through 2019: USE MyDB GO CREATE DATABASE SCOPED…
PolyBase and Windows Authentication in SQL Server 2022
Back in the SQL Server 2019 days, I laid out a post which covered a really annoying bug around Windows authentication and PolyBase. The upshot is that SQL Server 2019 CU2 broke Windows authentication, meaning that you could not create external objects on a SQL Server instance while using a Windows authenticated session. In CU2,…
PolyBase 2022 and Cosmos DB (Core API)
Getting Back to Basics To date, I've always used the MongoDB API for Cosmos DB when demonstrating PolyBase. This is true in PolyBase Revealed, as well as blog posts here. Today, that changes: we are going to use PolyBase to get data from a Cosmos DB using the Core API, also known as the SQL…
PolyBase 2022 and Cosmos DB (Mongo API)
Now that SQL Server 2022 has hit RC0, I've decided to give it a try and see what has changed with respect to PolyBase. In today's post, we'll look at connecting to Cosmos DB using the MongoDB PolyBase connector. The Problem Back in the 2019 days, I noted a problem when CU2 of SQL Server…
PolyBase versus the March of Time
PolyBase is dead: Today, we are announcing the retirement of PolyBase scale-out groups in Microsoft SQL Server. Scale-out group functionality will be removed from the product in SQL Server 2022. In-market SQL Server 2019, 2017, and 2016 will continue to support the functionality to the end of support for those products. Okay, not dead-dead. But let's talk…
PolyBase versus Flat Files
Wait, What? Today's post comes inspired by an Ed Pearson tweet that Chris Wood clued me into. https://twitter.com/ecpdata/status/1490496984544854016 Is there someone? Maybe! It took me a couple of weeks to get to this thanks to my crazy schedule but I have good news for you. PolyBase 12, Flat Files 8 I really like PolyBase's ability…
PolyBase and Postgres
Last month, I presented at year 4 of SQL Trail, a conference normally in Richmond, Virginia but entirely online this year due to obvious reasons. Before the conference, one of the attendees asked for a session on PolyBase, specifically around accessing Postgres data from SQL Server. I didn't have an example on that, so figured…
PolyBase and Excel: TOP Now Works
Back with SQL Server 2019 CU2, I reported an error with PolyBase connecting to Excel when trying to select TOP(10) from the table. I'm using the Microsoft Access Database Engine 2016 Redistributable's Excel driver. Here's a sample external data source and table statement: CREATE EXTERNAL DATA SOURCE VolcanoType WITH ( LOCATION = 'odbc://noplace', CONNECTION_OPTIONS =…
36 Chambers - The Legendary Journeys: Execution to the max! 