Jay Qi
@jayqi.com
Data Scientist for social good at drivendata.org, erstwhile aerospace engineer, intermittent user of social media • machine learning; data; open source; civic tech • www.jayqi.com
Great post! I appreciate the thorough exploration and discussion of different approaches.
Followup question: Pandas (as of version 2.0) can use PyArrow as a backend instead of numpy. Would using the list PyArrow data type address some of the shortcomings you identified for Pandas?
Followup question: Pandas (as of version 2.0) can use PyArrow as a backend instead of numpy. Would using the list PyArrow data type address some of the shortcomings you identified for Pandas?
February 26, 2025 at 12:33 AM
Great post! I appreciate the thorough exploration and discussion of different approaches.
Followup question: Pandas (as of version 2.0) can use PyArrow as a backend instead of numpy. Would using the list PyArrow data type address some of the shortcomings you identified for Pandas?
Followup question: Pandas (as of version 2.0) can use PyArrow as a backend instead of numpy. Would using the list PyArrow data type address some of the shortcomings you identified for Pandas?