Modin on LinkedIn: Pandas vs. SQL - Part 3: Pandas Is More Flexible (2024)

📣 As of TODAY, Ponder is in Public Beta. 🚀Now ANYONE can start running Python data workflows (pandas, NumPy) directly in their data warehouse for free in less than five minutes. Just pip install ponder.Learn more: https://lnkd.in/ge6jtMfb🦥 The problem: Because vanilla pandas operates in-memory, it grinds to a halt on medium-to-large datasets and doesn’t have the same security guarantees as your database.😫 The status quo “solution”: To productionize pandas code, teams rewrite it into other big data languages. Why?! Why do this to ourselves! No matter whether your data is small, medium, or large, the same code should run as is, without additional effort from the user.🥇 The better way: Ponder lets you run pandas, at scale and securely, in your data warehouse or database. It uses the pandas API and preserves the pandas UX, but under the hood, translates your pandas code to SQL and sends it to your data warehouse.🦾 The result: You get to code in pandas, but have your queries run on one of the most battle-tested and heavily-optimized infrastructures today — databases.Get started now at https://lnkd.in/gyTThiPg.#python #pandas #datascience

75

24 Comments

🚀 Attending #SnowflakeDevDay today? Join us this afternoon to learn how you can build an open-source AI stack with Modin, Streamlit, #Arctic, and more to power your LLM App! 🧑💻💡Find us at our Builder's Zone booth throughout the day to chat about Modin and all things Open Source! 🛠️🔍🗓️ Thursday, June 6 ⏰ 2:30 PM - 2:50 PM PDT 📍 Builders Hub Theater (Basecamp, South)#SnowflakeSummit #OpenSource #AI #LLM

9

1 Comment

Attending #SnowflakeSummit? ❄️ Visit our booth at Builder's Hub to explore Modin, talk to our Modin OSS contributors, and check out cool demos! See you there! 👋#python #datascience #machinelearning

12

We’re excited to announce Snowflake's intent to acquire Ponder to bring Ponder's Python data science innovations to all its customers.Thank you to all of the Ponder and Modin supporters who brought us to this point -- We're excited for what's to come!https://lnkd.in/gXmKnDNZ

28

Great coverage in the Real Python podcast of Mahesh Vashishtha's article: "Why Are There So Many Python Dataframes?"Here's the link to jump straight to that 7-minute portion of the podcast:https://lnkd.in/gBDw4TMMAnd here's the link to Mahesh's article: https://lnkd.in/gnjnSPmm#Python #dataframes

11

1 Comment

You can run NumPy 🔢 at scale in your database with Ponder 💥Through Modin, Ponder supports:- Element-wise matrix ops (addition, subtraction, multiplication, division, power)- Axis-collapsing or reducing ops (min, max, sum, product, mean)- Multi-array operations such as maximum or minimumAnd more: https://lnkd.in/gDjUsNMr#numpy #sql #database

31

2 Comments

If you're backed into a dark alley, facing a pack of ravenous NaN dogs 🐺, you'll want a sidekick like pandas dropna 👊 .You can drop rows with n missing values using df.dropna(thresh=n).You can drop entirely empty rows with df.dropna(how='all').You can drop rows missing a value in a particular column with df.dropna(subset='Column Name').Read 🐍 Matt Harrison's most recent Professional Pandas post to learn more: https://lnkd.in/gsMXxkuT#pandas #Python #dropna

15

6 Comments

A 2-hour pandas workflow (150 million rows) can take 2 minutes with Ponder on Snowflake.Watch the demo: https://lnkd.in/gFERWTTpOr read more: https://lnkd.in/gEbyxsTX#pandas #SQL #Snowflake

18

How do you use database connections and cursors in Python?It can be hard to figure out because database concepts and Python objects don't map 1:1.To learn more (with an emphasis on the Snowflake-Python connector) check out: https://lnkd.in/gf4s5QaN#python #database #snowflake

15