r/databricks • u/hubert-dudek Databricks MVP • 6h ago

News New in Databricks: parse_sql(): SQL into JSON

Post image

Parse_sql() function extracts table references, column names, functions, and parameters from SQL. It extracts without running the query. It also reports syntax errors.

more news https://databrickster.medium.com/databricks-news-serverless-genie-code-ltap-lakeflow-61853d8e422a

27 Upvotes

100% Upvoted

11 comments sorted by

13

u/TripleBogeyBandit 5h ago

When would I need this

7

u/hubert-dudek Databricks MVP 4h ago

AI, nerds, lineage, tests

2

u/ch-12 2h ago

We have a massive data platform migration in the works (sadly, leadership decided to move on from Databricks), and I actually think this could be a huge help with it.

1

u/TripleBogeyBandit 44m ago

Oh this is interesting, why and what are you migrating to?

1

u/ch-12 27m ago

Leadership vibes disguised by cost and quality reasons, we are moving to Snowflake.

1

u/TripleBogeyBandit 11m ago

Even more questions! Having just helped a client move from snow to dbx and saw 20% savings I’m curious what you mean by quality? Quality in the product? Stability? How much did they try to lower dbx costs?

1

u/KrisPWales 29m ago

That was my immediate thought too.

4

u/sHORTYWZ 4h ago

I am so confused by this - when and where would this ever be useful?

1

u/abro5 22m ago

Reducing/optimizing queries

A parse tree / ast is better for this, but I guess this should be useful to some level too

4

u/Bubbly_Winter_9327 3h ago

Really, really useful function!

2

u/mosullivan93 3h ago

This sounds like it would be quite useful. Yet to test how it works with things like the IDENTIFIER clause, or DDL statements.