Lightnews — Scholar-powered news

PySpark Data Engineering Scenario-Based Questions : : : : 1. Handling… | Karthik K.

PySpark Data Engineering Scenario-Based Questions : : : : 1. Handling Large Datasets How would you optimize a PySpark job that processes a large dataset (1TB+) to improve performance and reduce execut...

April 17, 2025 at 3:30 AM

akaicomet

@akaicomet.bsky.social

Data Engineer Interview Questions (2 to 5 Years of Experience) 1. What… | Karthik K.

Data Engineer Interview Questions (2 to 5 Years of Experience) 1. What are the key responsibilities of a Data Engineer? 2. Explain the difference between OLTP and OLAP systems. 3. What are ETL and EL...

www.linkedin.com/posts/activi...

April 17, 2025 at 2:50 AM

akaicomet

@akaicomet.bsky.social

Sign Up | LinkedIn

500 million+ members | Manage your professional identity. Build and engage with your professional network. Access knowledge, insights and opportunities.

www.linkedin.com/posts/bonthu...

April 11, 2025 at 5:42 AM

akaicomet

@akaicomet.bsky.social

#pyspark #dataengineering #sparkinterview | Bonthu Tarun Teja Reddy

𝐀𝐜𝐜𝐞𝐧𝐭𝐮𝐫𝐞 𝐏𝐲𝐒𝐩𝐚𝐫𝐤 𝐈𝐧𝐭𝐞𝐫𝐯𝐢𝐞𝐰 𝐐𝐮𝐞𝐬𝐭𝐢𝐨𝐧𝐬 𝐟𝐨𝐫 𝐃𝐚𝐭𝐚 𝐄𝐧𝐠𝐢𝐧𝐞𝐞𝐫 𝟐𝟎𝟐𝟓 1. Explain the key differences between Apache Spark's DataFrame and RDD APIs. In which scenarios would you prefer one over the other?...

April 11, 2025 at 2:32 AM

akaicomet

@akaicomet.bsky.social

Interview discussion on Spark: Interviewer: "Can you explain how you would… | Himanshu Jain

Interview discussion on Spark: Interviewer: "Can you explain how you would handle a large-scale data processing task with a tight deadline?" Candidate: "I would consider using Apache Sp...

www.linkedin.com/posts/rajash...

April 11, 2025 at 2:32 AM

akaicomet

@akaicomet.bsky.social

#pyspark #pysparkinterviewquestions #sparkinterview #apachespark… | Rajashekar .

𝐓𝐨𝐩 𝟏𝟎 𝐏𝐲𝐒𝐩𝐚𝐫𝐤 𝐒𝐜𝐞𝐧𝐚𝐫𝐢𝐨-𝐁𝐚𝐬𝐞𝐝 𝐈𝐧𝐭𝐞𝐫𝐯𝐢𝐞𝐰 𝐐𝐮𝐞𝐬𝐭𝐢𝐨𝐧𝐬 𝐟𝐨𝐫 𝐃𝐚𝐭𝐚 𝐄𝐧𝐠𝐢𝐧𝐞𝐞𝐫𝐬! 𝟏. 𝐇𝐚𝐧𝐝𝐥𝐢𝐧𝐠 𝐋𝐚𝐫𝐠𝐞 𝐃𝐚𝐭𝐚𝐬𝐞𝐭𝐬 How would you optimize a PySpark job that processes a large dataset (1TB+) to improve performance and...

April 11, 2025 at 2:31 AM

akaicomet

@akaicomet.bsky.social

#bigdata #dataengineer #dataanalytics #sql #sqlqueries | Himanshu Jain

𝐒𝐐𝐋 𝐂𝐎𝐀𝐋𝐄𝐒𝐂𝐄 𝐟𝐮𝐧𝐜𝐭𝐢𝐨𝐧 𝐕𝐬 𝐈𝐒𝐍𝐔𝐋𝐋 𝐟𝐮𝐧𝐜𝐭𝐢𝐨𝐧 𝐕𝐬 𝐂𝐀𝐒𝐄 𝐬𝐭𝐚𝐭𝐞𝐦𝐞𝐧𝐭𝐬: These are the important functions in case of analytical data generation, reporting. COALESCE, ISNULL are used to replace null values in res...

April 11, 2025 at 2:31 AM

akaicomet

@akaicomet.bsky.social

Daily issues faced by Data Engineers: 1. How do you handle job failures… | Karthik K.

Daily issues faced by Data Engineers: 1. How do you handle job failures in an ETL pipeline? 2. What steps do you take when a data pipeline is running slower than expected? 3. How do you address data ...

www.linkedin.com/posts/bonthu...

April 11, 2025 at 2:30 AM

akaicomet

@akaicomet.bsky.social

𝐇𝐂𝐋 𝐓𝐞𝐜𝐡𝐧𝐨𝐥𝐨𝐠𝐢𝐞𝐬 𝐃𝐚𝐭𝐚 𝐄𝐧𝐠𝐢𝐧𝐞𝐞𝐫… | Bonthu Tarun Teja Reddy

𝐇𝐂𝐋 𝐓𝐞𝐜𝐡𝐧𝐨𝐥𝐨𝐠𝐢𝐞𝐬 𝐃𝐚𝐭𝐚 𝐄𝐧𝐠𝐢𝐧𝐞𝐞𝐫 𝐈𝐧𝐭𝐞𝐫𝐯𝐢𝐞𝐰 - 𝐑𝐨𝐮𝐧𝐝 𝟏 𝐐𝐮𝐞𝐬𝐭𝐢𝐨𝐧𝐬 Introduction & Experience-Based Questions 1. Can you give us a brief introduction about yourself and your experience as a Data Enginee...

April 11, 2025 at 2:30 AM

akaicomet

@akaicomet.bsky.social

Processing 1 TB of Data in Databricks with Different Cluster… | Himanshu Jain

Processing 1 TB of Data in Databricks with Different Cluster Configurations: Interviewer: How long do you think it would take on a single machine without using any cluster? Candidate: On a single ma...

www.linkedin.com/posts/aditya...

April 11, 2025 at 2:29 AM

akaicomet

@akaicomet.bsky.social

SQL performance tuning interview questions asked in multiple MNC’s like… | Aditya Chandak | 24 comments

SQL performance tuning interview questions asked in multiple MNC’s like Sapient Tata Consultancy Services Accenture Deloitte and More! Basic Questions 1.⁠ ⁠What is SQL performance tuning, and why...

April 11, 2025 at 2:29 AM

akaicomet

@akaicomet.bsky.social

PySpark Basic Questions (0-2 Years Experience) 1. What is PySpark, and… | Karthik K.

PySpark Basic Questions (0-2 Years Experience) 1. What is PySpark, and how is it different from Apache Spark? 2. Explain the difference between RDD, DataFrame, and Dataset. 3. How do you create a Dat...

www.linkedin.com/posts/abhise...

April 8, 2025 at 4:29 PM

akaicomet

@akaicomet.bsky.social

80% of Candidates Get Rejected in the First 10 Minutes. Here's Why… | Abhisek Sahu | 131 comments

80% of Candidates Get Rejected in the First 10 Minutes. Here's Why : You walk into a data engineering interview You know Spark , Python, SQL You’ve used Airflow. You can explain Kafka. But...

www.linkedin.com/posts/abhije...

April 8, 2025 at 4:28 PM

akaicomet

@akaicomet.bsky.social

Part 1: Basic Concepts and Architecture 1. What is a stream in Snowflake,… | Abhijeet Ramteke

Part 1: Basic Concepts and Architecture 1. What is a stream in Snowflake, and what are the columns present in a stream? 2. What is the architecture of Snowflake? 3. What is a Snowpipe in the context ...

www.linkedin.com/posts/aditya...

April 8, 2025 at 4:27 PM

akaicomet

@akaicomet.bsky.social

#pyspark #dataengineer #azure #aws #gcp #etl #hadoop | Aditya Chandak | 18 comments

Data Engineer Scenario based interview! These scenarios asked in multiple MNC’s i.e. Accenture KPMG Deloitte Tata Consultancy Services Sapient and more!! Scenario 1: Interviewer: Can you design a da...

April 8, 2025 at 4:26 PM

akaicomet

@akaicomet.bsky.social

Data Engineer Interview Questions (2 to 5 Years of Experience) 1. What… | Karthik K.

Data Engineer Interview Questions (2 to 5 Years of Experience) 1. What are the key responsibilities of a Data Engineer? 2. Explain the difference between OLTP and OLAP systems. 3. What are ETL and EL...

April 8, 2025 at 4:25 PM

akaicomet

@akaicomet.bsky.social

Capgemini SQL interview questions for Round 1 (for 3 to 5 years of… | Karthik K.

Capgemini SQL interview questions for Round 1 (for 3 to 5 years of experience) 1. Explain the differences between DELETE, TRUNCATE, and DROP in SQL. In what scenarios would you use each of them? 2....

www.linkedin.com/posts/ajay02...

April 7, 2025 at 2:05 AM

akaicomet

@akaicomet.bsky.social

#dataengineering #sql #pipelinedesign #techinterviews #kafka #spark… | Ajay Kadiyala | 65 comments

"I Failed This Data Engineering Interview—But You Don’t Have To"...... 🧠 𝗦𝗤𝗟 & 𝗗𝗮𝘁𝗮 𝗠𝗼𝗱𝗲𝗹𝗶𝗻𝗴 1. “How would you find the first non-repeating value in a column using SQL?” 2. “What’s your...

April 7, 2025 at 2:04 AM

akaicomet

@akaicomet.bsky.social

12 𝐦𝐨𝐬𝐭 𝐜𝐨𝐦𝐦𝐨𝐧 𝐆𝐢𝐭 𝐜𝐨𝐦𝐦𝐚𝐧𝐝𝐬! 1. git init Purpose:… | Himanshu Jain

12 𝐦𝐨𝐬𝐭 𝐜𝐨𝐦𝐦𝐨𝐧 𝐆𝐢𝐭 𝐜𝐨𝐦𝐦𝐚𝐧𝐝𝐬! 1. git init Purpose: Initializes a new Git repository in your current directory. Usage: Run git init to start tracking your project with Git 2. git clone Purpose: Creates ...

April 7, 2025 at 2:04 AM

akaicomet

@akaicomet.bsky.social

Daily issues faced by Data Engineers: 1. How do you handle job failures… | Karthik K.

Daily issues faced by Data Engineers: 1. How do you handle job failures in an ETL pipeline? 2. What steps do you take when a data pipeline is running slower than expected? 3. How do you address data ...

www.linkedin.com/posts/sailaj...

April 7, 2025 at 2:03 AM

akaicomet

@akaicomet.bsky.social

#pyspark #python #interviewtips #interviewquestions | Sailaja Mendu

𝐌𝐨𝐬𝐭 𝐀𝐬𝐤𝐞𝐝 𝐏𝐲𝐒𝐩𝐚𝐫𝐤 𝐈𝐧𝐭𝐞𝐫𝐯𝐢𝐞𝐰 𝐐𝐮𝐞𝐬𝐭𝐢𝐨𝐧𝐬 (𝟐𝟎𝟐𝟓) . . . . . . . . . . . . 𝐆𝐞𝐧𝐞𝐫𝐚𝐥 𝐐𝐮𝐞𝐬𝐭𝐢𝐨𝐧𝐬 1. What is PySpark, and how does it work? 2. Explain the architecture of PySpark. 3. What are the differences ...

www.linkedin.com/posts/manoj-...

April 6, 2025 at 2:30 AM

akaicomet

@akaicomet.bsky.social

#awsglue #pyspark #dataengineering #etloptimization #bigdata #aws | Manoj kumaar C

🚀 Optimizing Data Reading in AWS Glue: Spark SQL vs. PySpark After rigorous testing with AWS Glue jobs, here’s my verdict on which approach delivers better performance for column pruning and predicat...

www.linkedin.com/posts/mohamm...

April 6, 2025 at 2:29 AM

akaicomet

@akaicomet.bsky.social

#mastering #dataengineering #apachespark #sparksql #bigdata #etl #pyspark… | Mohammad Nazim

#Mastering SQL in Apache Spark: The Secret Sauce for Scalable Data Engineering! __________________________________________ . . . . . Whether you’re building ETL pipelines or running analytics on bil...

April 6, 2025 at 2:27 AM

akaicomet

@akaicomet.bsky.social

Tech Mahindra SQL interview questions for a Data Engineer (3-5 years of… | Karthik K.

Tech Mahindra SQL interview questions for a Data Engineer (3-5 years of experience. ) 1. Find the top 3 cities with the highest sales per month. (sales_table): | sale_id | city | sale_date | amo...